Invariant Properties

  • rss
  • Home

Creating Password-Based Encryption Keys

Bear Giles | September 29, 2013

This article discusses creating password-based encryption PBE keys.

First a reminder of earlier points – as a rule you should, when practical, use the PBE key as a master key that is used solely to unlock a working key. This has three major benefits:

  • You can have multiple passwords, e.g., an escrowed recovery key,
  • You can change your password without being forced to reencrypt everything,
  • You can change your working key without being forced to change your password.

I’ll discuss using working keys in database encryption in a future article.

Password-Based Encryption Key Generation with PBKDF2WithHmacSHA1

Java has not had a standard way to create a PBE Key in the past. Individual cryptographic providers provided their own generators but it was a painful process to identify and use a generator that matched your cipher.

This changed with Java 7. There is now a standard key generation algorithm that is available in all JCE implementations. It can definitely be used to produce AES keys. I’ve seen an example of it being used to produce a key of arbitrary length but I haven’t been able to duplicate that behavior – it may be a nonstandard extension.

This algorithm takes four parameters. The first is the key length – use 128 for AES keys. Other possible values are 192 and 256 bits. The second is the number of iterations. Your wifi router uses 4096 iterations but many people now recommend at least 10,000 iterations.

The third parameter is the ‘salt’. A wifi router uses the SSID, many sites use a small file, and I’ll discuss another approach below. The salt should be large enough that the entropy is greater than the key length. E.g., if you want a 128-bit key you should either have (at least) 128 bits of random binary data or about 22 random alpha-numeric characters.

The final parameter is the password. Again the entropy should be greater than the key length. In a webapp the password is often provided by the app server via JNDI.

Finally we want both a cipher key and an IV, not just a cipher key. The lack of an IV, or the use of a weak one, is one of the most common mistakes made by people unfamiliar with cryptography. (See: Not using a random initialization vector with cipher block chaining mode [owasp.org].) One common approach is to generate a random salt and prepend it to the ciphertext for use during decryption but I’ll discuss a different approach that uses the password and salt.

Now the code. First we see how to create a cipher key and IV from a password and salt. (We’ll discuss the salt in a moment.)

  1. public class PbkTest {
  2.     private static final Provider bc = new BouncyCastleProvider();
  3.     private static final ResourceBundle BUNDLE = ResourceBundle
  4.             .getBundle(PbkTest.class.getName());
  5.     private SecretKey cipherKey;
  6.     private AlgorithmParameterSpec ivSpec;
  7.  
  8.     /**
  9.      * Create secret key and IV from password.
  10.      *
  11.      * Implementation note: I've believe I've seen other code that can extract
  12.      * the random bits for the IV directly from the PBEKeySpec but I haven't
  13.      * been able to duplicate it. It might have been a BouncyCastle extension.
  14.      *
  15.      * @throws Exception
  16.      */
  17.     public void createKeyAndIv(char[] password) throws SecurityException,
  18.             NoSuchAlgorithmException, InvalidKeySpecException {
  19.         final String algorithm = "PBKDF2WithHmacSHA1";
  20.         final SecretKeyFactory factory = SecretKeyFactory
  21.                 .getInstance(algorithm);
  22.         final int derivedKeyLength = 128;
  23.         final int iterations = 10000;
  24.  
  25.         // create salt
  26.         final byte[][] salt = feistelSha1Hash(createSalt(), 1000);
  27.  
  28.         // create cipher key
  29.         final PBEKeySpec cipherSpec = new PBEKeySpec(password, salt[0],
  30.                 iterations, derivedKeyLength);
  31.         cipherKey = factory.generateSecret(cipherSpec);
  32.         cipherSpec.clearPassword();
  33.  
  34.         // create IV. This is just one of many approaches. You do
  35.         // not want to use the same salt used in creating the PBEKey.
  36.         try {
  37.             final Cipher cipher = Cipher.getInstance("AES/CBC/NoPadding", bc);
  38.             cipher.init(Cipher.ENCRYPT_MODE, cipherKey, new IvParameterSpec(
  39.                     salt[1], 0, 16));
  40.             ivSpec = new IvParameterSpec(cipher.doFinal(salt[1], 4, 16));
  41.         } catch (NoSuchPaddingException e) {
  42.             throw new SecurityException("unable to create IV", e);
  43.         } catch (InvalidAlgorithmParameterException e) {
  44.             throw new SecurityException("unable to create IV", e);
  45.         } catch (InvalidKeyException e) {
  46.             throw new SecurityException("unable to create IV", e);
  47.         } catch (BadPaddingException e) {
  48.             throw new SecurityException("unable to create IV", e);
  49.         } catch (IllegalBlockSizeException e) {
  50.             throw new SecurityException("unable to create IV", e);
  51.         }
  52.     }
  53. }
public class PbkTest {
    private static final Provider bc = new BouncyCastleProvider();
    private static final ResourceBundle BUNDLE = ResourceBundle
            .getBundle(PbkTest.class.getName());
    private SecretKey cipherKey;
    private AlgorithmParameterSpec ivSpec;

    /**
     * Create secret key and IV from password.
     * 
     * Implementation note: I've believe I've seen other code that can extract
     * the random bits for the IV directly from the PBEKeySpec but I haven't
     * been able to duplicate it. It might have been a BouncyCastle extension.
     * 
     * @throws Exception
     */
    public void createKeyAndIv(char[] password) throws SecurityException,
            NoSuchAlgorithmException, InvalidKeySpecException {
        final String algorithm = "PBKDF2WithHmacSHA1";
        final SecretKeyFactory factory = SecretKeyFactory
                .getInstance(algorithm);
        final int derivedKeyLength = 128;
        final int iterations = 10000;

        // create salt
        final byte[][] salt = feistelSha1Hash(createSalt(), 1000);

        // create cipher key
        final PBEKeySpec cipherSpec = new PBEKeySpec(password, salt[0],
                iterations, derivedKeyLength);
        cipherKey = factory.generateSecret(cipherSpec);
        cipherSpec.clearPassword();

        // create IV. This is just one of many approaches. You do
        // not want to use the same salt used in creating the PBEKey.
        try {
            final Cipher cipher = Cipher.getInstance("AES/CBC/NoPadding", bc);
            cipher.init(Cipher.ENCRYPT_MODE, cipherKey, new IvParameterSpec(
                    salt[1], 0, 16));
            ivSpec = new IvParameterSpec(cipher.doFinal(salt[1], 4, 16));
        } catch (NoSuchPaddingException e) {
            throw new SecurityException("unable to create IV", e);
        } catch (InvalidAlgorithmParameterException e) {
            throw new SecurityException("unable to create IV", e);
        } catch (InvalidKeyException e) {
            throw new SecurityException("unable to create IV", e);
        } catch (BadPaddingException e) {
            throw new SecurityException("unable to create IV", e);
        } catch (IllegalBlockSizeException e) {
            throw new SecurityException("unable to create IV", e);
        }
    }
}

We could simply load a file containing random binary data but using a Feistel cipher allows us to mix entropy from two sources.

  1.     /**
  2.      * Create salt. Two values are provided to support creation of both a cipher
  3.      * key and IV from a single password.
  4.      *
  5.      * The 'left' salt is pulled from a file outside of the app context. this
  6.      * makes it much harder for a compromised app to obtain or modify this
  7.      * value. You could read it as classloader resource but that's not really
  8.      * different from the properties file used below. Another possibility is to
  9.      * load it from a read-only value in a database, ideally one with a
  10.      * different schema than the rest of the application. (It could even be an
  11.      * in-memory database such as H2 that contains nothing but keying material,
  12.      * again initialized from a file outside of the app context.)
  13.      *
  14.      * The 'right' salt is pulled from a properties file. It is possible to use
  15.      * a base64-encoded value but administration is a lot easier if we just take
  16.      * an arbitrary string and hash it ourselves. At a minimum it should be a
  17.      * random mix-cased string of at least (120/5 = 24) characters.
  18.      *
  19.      * The generated salts are equally strong.
  20.      *
  21.      * Implementation note: since this is for demonstration purposes a static
  22.      * string in used in place of reading an external file.
  23.      */
  24.     public byte[][] createSalt() throws NoSuchAlgorithmException {
  25.         final MessageDigest digest = MessageDigest.getInstance("SHA1");
  26.         final byte[] left = new byte[20]; // fall back to all zeroes
  27.         final byte[] right = new byte[20]; // fall back to all zeroes
  28.  
  29.         // load value from file or database.
  30.         // note: we use fixed value for demonstration purposes.
  31.         final String leftValue = "this string should be read from file or database";
  32.         if (leftValue != null) {
  33.             System.arraycopy(digest.digest(leftValue.getBytes()), 0, left, 0,
  34.                     left.length);
  35.             digest.reset();
  36.         }
  37.  
  38.         // load value from resource bundle.
  39.         final String rightValue = BUNDLE.getString("salt");
  40.         if (rightValue != null) {
  41.             System.arraycopy(digest.digest(rightValue.getBytes()), 0, right, 0,
  42.                     right.length);
  43.             digest.reset();
  44.         }
  45.  
  46.         final byte[][] salt = feistelSha1Hash(new byte[][] { left, right },
  47.                 1000);
  48.  
  49.         return salt;
  50.     }
    /**
     * Create salt. Two values are provided to support creation of both a cipher
     * key and IV from a single password.
     * 
     * The 'left' salt is pulled from a file outside of the app context. this
     * makes it much harder for a compromised app to obtain or modify this
     * value. You could read it as classloader resource but that's not really
     * different from the properties file used below. Another possibility is to
     * load it from a read-only value in a database, ideally one with a
     * different schema than the rest of the application. (It could even be an
     * in-memory database such as H2 that contains nothing but keying material,
     * again initialized from a file outside of the app context.)
     * 
     * The 'right' salt is pulled from a properties file. It is possible to use
     * a base64-encoded value but administration is a lot easier if we just take
     * an arbitrary string and hash it ourselves. At a minimum it should be a
     * random mix-cased string of at least (120/5 = 24) characters.
     * 
     * The generated salts are equally strong.
     * 
     * Implementation note: since this is for demonstration purposes a static
     * string in used in place of reading an external file.
     */
    public byte[][] createSalt() throws NoSuchAlgorithmException {
        final MessageDigest digest = MessageDigest.getInstance("SHA1");
        final byte[] left = new byte[20]; // fall back to all zeroes
        final byte[] right = new byte[20]; // fall back to all zeroes

        // load value from file or database.
        // note: we use fixed value for demonstration purposes.
        final String leftValue = "this string should be read from file or database";
        if (leftValue != null) {
            System.arraycopy(digest.digest(leftValue.getBytes()), 0, left, 0,
                    left.length);
            digest.reset();
        }

        // load value from resource bundle.
        final String rightValue = BUNDLE.getString("salt");
        if (rightValue != null) {
            System.arraycopy(digest.digest(rightValue.getBytes()), 0, right, 0,
                    right.length);
            digest.reset();
        }

        final byte[][] salt = feistelSha1Hash(new byte[][] { left, right },
                1000);

        return salt;
    }

A practical implementation using both a resource bundle (which is visible in the classpath) and a string loaded from the filesystem or database is:

  1.     /**
  2.      * Create salt. Two values are provided to support creation of both a cipher
  3.      * key and IV from a single password.
  4.      *
  5.      * The 'left' salt is pulled from a file outside of the app context. this
  6.      * makes it much harder for a compromised app to obtain or modify this
  7.      * value. You could read it as classloader resource but that's not really
  8.      * different from the properties file used below. Another possibility is to
  9.      * load it from a read-only value in a database, ideally one with a
  10.      * different schema than the rest of the application. (It could even be an
  11.      * in-memory database such as H2 that contains nothing but keying material,
  12.      * again initialized from a file outside of the app context.)
  13.      *
  14.      * The 'right' salt is pulled from a properties file. It is possible to use
  15.      * a base64-encoded value but administration is a lot easier if we just take
  16.      * an arbitrary string and hash it ourselves. At a minimum it should be a
  17.      * random mix-cased string of at least (120/5 = 24) characters.
  18.      *
  19.      * The generated salts are equally strong.
  20.      *
  21.      * Implementation note: since this is for demonstration purposes a static
  22.      * string in used in place of reading an external file.
  23.      */
  24.     public byte[][] createSalt() throws NoSuchAlgorithmException {
  25.         final MessageDigest digest = MessageDigest.getInstance("SHA1");
  26.         final byte[] left = new byte[20]; // fall back to all zeroes
  27.         final byte[] right = new byte[20]; // fall back to all zeroes
  28.  
  29.         // load value from file or database.
  30.         // note: we use fixed value for demonstration purposes.
  31.         final String leftValue = "this string should be read from file or database";
  32.         if (leftValue != null) {
  33.             System.arraycopy(digest.digest(leftValue.getBytes()), 0, left, 0,
  34.                     left.length);
  35.             digest.reset();
  36.         }
  37.  
  38.         // load value from resource bundle.
  39.         final String rightValue = BUNDLE.getString("salt");
  40.         if (rightValue != null) {
  41.             System.arraycopy(digest.digest(rightValue.getBytes()), 0, right, 0,
  42.                     right.length);
  43.             digest.reset();
  44.         }
  45.  
  46.         final byte[][] salt = feistelSha1Hash(new byte[][] { left, right },
  47.                 1000);
  48.  
  49.         return salt;
  50.     }
    /**
     * Create salt. Two values are provided to support creation of both a cipher
     * key and IV from a single password.
     * 
     * The 'left' salt is pulled from a file outside of the app context. this
     * makes it much harder for a compromised app to obtain or modify this
     * value. You could read it as classloader resource but that's not really
     * different from the properties file used below. Another possibility is to
     * load it from a read-only value in a database, ideally one with a
     * different schema than the rest of the application. (It could even be an
     * in-memory database such as H2 that contains nothing but keying material,
     * again initialized from a file outside of the app context.)
     * 
     * The 'right' salt is pulled from a properties file. It is possible to use
     * a base64-encoded value but administration is a lot easier if we just take
     * an arbitrary string and hash it ourselves. At a minimum it should be a
     * random mix-cased string of at least (120/5 = 24) characters.
     * 
     * The generated salts are equally strong.
     * 
     * Implementation note: since this is for demonstration purposes a static
     * string in used in place of reading an external file.
     */
    public byte[][] createSalt() throws NoSuchAlgorithmException {
        final MessageDigest digest = MessageDigest.getInstance("SHA1");
        final byte[] left = new byte[20]; // fall back to all zeroes
        final byte[] right = new byte[20]; // fall back to all zeroes

        // load value from file or database.
        // note: we use fixed value for demonstration purposes.
        final String leftValue = "this string should be read from file or database";
        if (leftValue != null) {
            System.arraycopy(digest.digest(leftValue.getBytes()), 0, left, 0,
                    left.length);
            digest.reset();
        }

        // load value from resource bundle.
        final String rightValue = BUNDLE.getString("salt");
        if (rightValue != null) {
            System.arraycopy(digest.digest(rightValue.getBytes()), 0, right, 0,
                    right.length);
            digest.reset();
        }

        final byte[][] salt = feistelSha1Hash(new byte[][] { left, right },
                1000);

        return salt;
    }

Finally we can see it in practice in two test methods:

  1.     /**
  2.      * Obtain password. Architectually we'll want good "separation of concerns"
  3.      * and we should get the cipher key and IV from a separate place than where
  4.      * we use it.
  5.      *
  6.      * This is a unit test so the password is stored in a properties file. In
  7.      * practice we'll want to get it from JNDI from an appserver, or at least a
  8.      * file outside of the appserver's directory.
  9.      *
  10.      * @throws Exception
  11.      */
  12.     @Before
  13.     public void setup() throws Exception {
  14.         createKeyAndIv(BUNDLE.getString("password").toCharArray());
  15.     }
  16.  
  17.     /**
  18.      * Test encryption.
  19.      *
  20.      * @throws Exception
  21.      */
  22.     @Test
  23.     public void testEncryption() throws Exception {
  24.         String plaintext = BUNDLE.getString("plaintext");
  25.  
  26.         Cipher cipher = Cipher.getInstance(BUNDLE.getString("algorithm"), bc);
  27.         cipher.init(Cipher.ENCRYPT_MODE, cipherKey, ivSpec);
  28.         byte[] actual = cipher.doFinal(plaintext.getBytes());
  29.         assertEquals(BUNDLE.getString("ciphertext"),
  30.                 new String(Base64.encode(actual), Charset.forName("UTF-8")));
  31.     }
  32.  
  33.     /**
  34.      * Test decryption.
  35.      *
  36.      * @throws Exception
  37.      */
  38.     @Test
  39.     public void testEncryptionAndDecryption() throws Exception {
  40.         String ciphertext = BUNDLE.getString("ciphertext");
  41.  
  42.         Cipher cipher = Cipher.getInstance(BUNDLE.getString("algorithm"), bc);
  43.         cipher.init(Cipher.DECRYPT_MODE, cipherKey, ivSpec);
  44.         byte[] actual = cipher.doFinal(Base64.decode(ciphertext));
  45.  
  46.         assertEquals(BUNDLE.getString("plaintext"),
  47.                 new String(actual, Charset.forName("UTF-8")));
  48.     }
    /**
     * Obtain password. Architectually we'll want good "separation of concerns"
     * and we should get the cipher key and IV from a separate place than where
     * we use it.
     * 
     * This is a unit test so the password is stored in a properties file. In
     * practice we'll want to get it from JNDI from an appserver, or at least a
     * file outside of the appserver's directory.
     * 
     * @throws Exception
     */
    @Before
    public void setup() throws Exception {
        createKeyAndIv(BUNDLE.getString("password").toCharArray());
    }

    /**
     * Test encryption.
     * 
     * @throws Exception
     */
    @Test
    public void testEncryption() throws Exception {
        String plaintext = BUNDLE.getString("plaintext");

        Cipher cipher = Cipher.getInstance(BUNDLE.getString("algorithm"), bc);
        cipher.init(Cipher.ENCRYPT_MODE, cipherKey, ivSpec);
        byte[] actual = cipher.doFinal(plaintext.getBytes());
        assertEquals(BUNDLE.getString("ciphertext"),
                new String(Base64.encode(actual), Charset.forName("UTF-8")));
    }

    /**
     * Test decryption.
     * 
     * @throws Exception
     */
    @Test
    public void testEncryptionAndDecryption() throws Exception {
        String ciphertext = BUNDLE.getString("ciphertext");

        Cipher cipher = Cipher.getInstance(BUNDLE.getString("algorithm"), bc);
        cipher.init(Cipher.DECRYPT_MODE, cipherKey, ivSpec);
        byte[] actual = cipher.doFinal(Base64.decode(ciphertext));

        assertEquals(BUNDLE.getString("plaintext"),
                new String(actual, Charset.forName("UTF-8")));
    }

The full source code is available at http://code.google.com/p/invariant-properties-blog/source/browse/pbekey.

See also: NIST SP 800-132, Recommendation for Password-Based Key Derivation, section 5.3.

See also: http://stackoverflow.com/questions/2465690/pbkdf2-hmac-sha1/2465884#2465884 for discussion of creation of master key for WPA2 network.

Categories
java, security
Comments rss
Comments rss
Trackback
Trackback

« Using the JIRA REST java client: Comments and Attachments Spring-injected Beans in JPA EntityListeners »

Leave a Reply

Click here to cancel reply.

You must be logged in to post a comment.

Archives

  • May 2020 (1)
  • March 2019 (1)
  • August 2018 (1)
  • May 2018 (1)
  • February 2018 (1)
  • November 2017 (4)
  • January 2017 (3)
  • June 2016 (1)
  • May 2016 (1)
  • April 2016 (2)
  • March 2016 (1)
  • February 2016 (3)
  • January 2016 (6)
  • December 2015 (2)
  • November 2015 (3)
  • October 2015 (2)
  • August 2015 (4)
  • July 2015 (2)
  • June 2015 (2)
  • January 2015 (1)
  • December 2014 (6)
  • October 2014 (1)
  • September 2014 (2)
  • August 2014 (1)
  • July 2014 (1)
  • June 2014 (2)
  • May 2014 (2)
  • April 2014 (1)
  • March 2014 (1)
  • February 2014 (3)
  • January 2014 (6)
  • December 2013 (13)
  • November 2013 (6)
  • October 2013 (3)
  • September 2013 (2)
  • August 2013 (5)
  • June 2013 (1)
  • May 2013 (2)
  • March 2013 (1)
  • November 2012 (1)
  • October 2012 (3)
  • September 2012 (2)
  • May 2012 (6)
  • January 2012 (2)
  • December 2011 (12)
  • July 2011 (1)
  • June 2011 (2)
  • May 2011 (5)
  • April 2011 (6)
  • March 2011 (4)
  • February 2011 (3)
  • October 2010 (6)
  • September 2010 (8)

Recent Posts

  • 8-bit Breadboard Computer: Good Encapsulation!
  • Where are all the posts?
  • Better Ad Blocking Through Pi-Hole and Local Caching
  • The difference between APIs and SPIs
  • Hadoop: User Impersonation with Kerberos Authentication

Meta

  • Log in
  • Entries RSS
  • Comments RSS
  • WordPress.org

Pages

  • About Me
  • Notebook: Common XML Tasks
  • Notebook: Database/Webapp Security
  • Notebook: Development Tips

Syndication

Java Code Geeks

Know Your Rights

Support Bloggers' Rights
Demand Your dotRIGHTS

Security

  • Dark Reading
  • Krebs On Security Krebs On Security
  • Naked Security Naked Security
  • Schneier on Security Schneier on Security
  • TaoSecurity TaoSecurity

Politics

  • ACLU ACLU
  • EFF EFF

News

  • Ars technica Ars technica
  • Kevin Drum at Mother Jones Kevin Drum at Mother Jones
  • Raw Story Raw Story
  • Tech Dirt Tech Dirt
  • Vice Vice

Spam Blocked

53,793 spam blocked by Akismet
rss Comments rss valid xhtml 1.1 design by jide powered by Wordpress get firefox