Invariant Properties

I recently had to add database encryption to a few fields and discovered a lot of bad advice out there.

Update

I recently learned how to handle spring injection into JPA listeners. Details at http://invariantproperties.com/2013/09/29/spring-injected-beans-in-jpa-entitylisteners/. This small project includes a variant of the classes discussed below.

Architectural Issues

The biggest problem is architectural. If your persistence manager quietly handles your encryption then, by definition, your architecture demands a tight and unnecessary binding between your persistence and security designs. You can’t touch one without also touching the other.

This might seem to be unavoidable but there is a respected school of thought that the best architecture is one where you have independent teams of app developers and security developers. The app developers can’t be sloppy but overall their sole focus is feature completion. The security developers are responsible for designing and implementing the security. The only places where both pieces are considered is at the architectural and top-level design.

In the past this wasn’t very practical but Aspect-Oriented Programming (AOP) and similar concepts have changed that. Now it’s entirely reasonable to inject an interceptor between the service and persistence layer so values the caller isn’t authorized to see are quietly dropped. A list of 10 items might be cut to 7, or an update might throw an exception instead of modifying a read-only value. It’s a little more complicated when persisting collections but the general approach should be clear.

The key thing here is that the app developers never need to see the security code. It can all be handled by AOP injection added via configuration files at the time of deployment. More importantly it can be changed at any time without requiring modification of the application itself. (You may need to perform an update process that will change values in the database.)

An interceptor can even prevent calls to undocumented methods – one less worry about rogue programmers.

In practice many sites will have several developers wear both hats instead of having a dedicated security team. That’s not a problem as long as they can keep their distinct responsibilities in mind.

Transparent encryption in JPA or Hibernate fields is definitely better than putting the encryption/decryption code in your POJO but it still imposes a high level of unnecessary binding between the security and persistence layers. It also has serious security issues.

Security Issues

There is a critical question any time you’re dealing with encryption – can this object be written to disk? The most obvious threat is serialization, e.g., by an appserver that is passivating data to free up memory or to migrate it to a different server.

In practice this means that your keys and plaintext content must be marked ‘transient’ (for the serialization engine) and ‘@Transient’ (for JPA or Hibernate). If you’re really paranoid you’ll even override the implicit serialization method writeObject so you can absolutely guarantee that these fields are never written to disk.

This works… but it blows the transparent encryption/decryption out of the water since the entire point of that code is to make these fields look like just another field. You must maintain two fields – a persistent encrypted value and a transient unencrypted value – and have some way to keep them in sync. All done without putting any crypto code into your pojo.

A more subtle problem is that your objects may still be written to disk if an attacker can trigger a core dump by crashing the appserver. Careful site administrators will disable core dumps but many overlook it. It’s harder to work around this but it’s possible if the AOP decrypts/encrypts values immediately around the methods that need the decrypted values. Your application shouldn’t care where the decryption occurs as long as it’s decrypted when it needs it. This is the type of decision that should be left to the security team.

A third way objects can be written to disk is via OS swap files but that should be a non-issue since swap files are usually encrypted now.

JPA EntityListeners

A solution is JPA EntityListeners or the corresponding Hibernate class. These are Listener classes that can provide methods called before or after database object creation, deletion or modification.

Sample Code

This is easiest to see with some sample code. Consider a situation where we must keep a user’s password for a third-party site. In this case we must use encryption, not hashing.

(Note: I doubt this is the actual information required by Twitter for third-party apps – it’s solely for the purpose of illustration.)

The entity

/**
 * Conventional POJO. Following other conventions the sensitive
 * information is written to a secondary table in addition to being
 * encrypted.
 */
@Entity
@Table(name="twitter")
@SecondaryTable(name="twitter_pw", pkJoinColumns=@PrimaryKeyJoinColumn(name="twitter_id"))
@EntityListeners(TwitterUserPasswordListener.class)
public class TwitterUser {
   private Integer id;
   private String twitterUser
   private String encryptedPassword;
   transient private String password;

   @Id
   @GeneratedValue(strategy = GenerationType.IDENTITY)
   public Integer getId() { return id; }

   @Column(name = "twitter_user")
   public String getTwitterUser() { return twitterUser; }

   @Column(name = "twitter_pw", table = "twitter_pw")
   @Lob
   public String getEncryptedPassword() { return encryptedPassword; }

   @Transient
   public String getPassword() { return password; }

   // similar definitions for setters....
}

The DAO

/**
 * Conventional DAO to access login information.
 */
@LocalBean
@Stateless
public class TwitterDao {
   @PersistenceContext
   private EntityManager em;

   /**
    * Read an object from the database.
    */
   @TransactionAttribute(TransactionAttributeType.SUPPORTS)
   public TwitterUser getUserById(Integer id) {
      return em.find(TwitterUser.class, id);
   }

   /**
    * Create a new record in the database.
    */
   @TransactionAttribute(TransactionAttributeType.REQUIRED)
   public saveTwitterUser(TwitterUser user) {
      em.persist(user);
   }

   /**
    * Update an existing record in the database.
    *
    * Note: this method uses JPA semantics. The Hibernate
    * saveOrUpdate() method uses slightly different semantics
    * but the required changes are straightforward.
    */
   @TransactionAttribute(TransactionAttributeType.REQUIRED)
   public updateTwitterUser(TwitterUser user) {
      TwitterUser tw = em.merge(user);

      // we need to make one change from the standard method -
      // during a 'merge' the old data read from the database
      // will result in the decrypted value overwriting the new
      // plaintext value - changes won't be persisted! This isn't
      // a problem when the object is eventually evicted from
      // the JPA/Hibernate cache so we're fine as long as we
      // explicitly copy any fields that are hit by the listener.
      tw.setPassword(user.getPassword());

      return tw;
   }

The EntityListener

To keep a clean separation between the persistence and security layers the listener does nothing but call a service that handles the encryption. It is completely ignorant of the encryption details.

public class TwitterUserPasswordListener {
   @Inject
   private EncryptorBean encryptor;

   /**
    * Decrypt password after loading.
    */
   @PostLoad
   @PostUpdate
   public void decryptPassword(Object pc) {
      if (!(pc instanceof TwitterUser)) {
         return;
      }

      TwitterUser user = (TwitterUser) pc;
      user.setPassword(null);

      if (user.getEncryptedPassword() != null) {
         user.setPassword(
            encryptor.decryptString(user.getEncryptedPassword());
      }
   }

   /**
    * Decrypt password before persisting
    */
   @PrePersist
   @PreUpdate
   public void encryptPassword(Object pc) {
      if (!(pc instanceof TwitterUser)) {
         return;
      }

      TwitterUser user = (TwitterUser) pc;
      user.setEncryptedPassword(null);

      if (user.getPassword() != null) {
         user.setEncryptedPassword(
            encryptor.encryptString(user.getPassword());
      }
   }
}

The EncryptorBean

The EncryptorBean handles encryption but does not know what’s being encrypted. This is a minimal implementation – in practice we’ll probably want to pass a keyId in addition to the ciphertext/plaintext. This would allow us to quietly rotate encryption keys with minimal disruption – something that is definitely not possible with the usual ‘easy encryption’ approaches.

This class uses OWASP/ESAPI for encryption since 1) it should already be used by your application and 2) the portable format allows other applications to use our database as long as they also use the OWASP/ESAPI library.

The implementation only covers Strings – a robust solution should have methods for all primitive types and possibly domain-specific classes such as credit cards.

import org.owasp.esapi.ESAPI;
import org.owasp.esapi.Encryptor;
import org.owasp.esapi.codecs.Base64;
import org.owasp.esapi.crypto.CipherText;
import org.owasp.esapi.crypto.PlainText;
import org.owasp.esapi.errors.EncryptionException;
import org.owasp.esapi.reference.crypto.JavaEncryptor;

@Stateless
public class EncryptorBean {
   private static final String PBE_ALGORITHM = "PBEWITHSHA256AND128BITAES-CBC-BC";
   private static final String ALGORITHM = "AES";

   // hardcoded for demonstration use. In production you might get the
   // salt from the filesystem and the password from a appserver JNDI value.
   private static final String SALT = "WR9bdtN3tMHg75PDK9PoIQ==";
   private static final char[] PASSWORD = "password".toCharArray();

   // the key
   private transient SecretKey key;

   /**
    * Constructor creates secret key. In production we may want
    * to avoid keeping the secret key hanging around in memory for
    * very long.
    */
   public EncryptorBean() {
      try {
         // create the PBE key
         KeySpec spec = new PBEKeySpec(PASSWORD, Base64.decode(SALT), 1024);
         SecretKey skey = SecretKeyFactory.getInstance(PBE_ALGORITHM).generateSecret(spec);
         // recast key as straightforward AES without padding.
         key = new SecretKeySpec(skey.getEncoded(), ALGORITHM);
      } catch (SecurityException ex) {
         // handle appropriately...
      }
   }

   /**
    * Decrypt String
    */
   public String decryptString(String ciphertext) {
      String plaintext = null;

      if (ciphertext != null) {
         try {
            Encryptor encryptor = JavaEncryptor.getInstance();
            CipherText ct = CipherText.from PortableSerializedBytes(Base64.decode(ciphertext));
            plaintext = encryptor.decrypt(key, ct).toString();
         } catch (EncryptionException e) {
            // handle exception. Perhaps set value to null?
         }
      }

      return plaintext;
   }

   /**
    * Encrypt String
    */
   public String encryptString(String plaintext) {
      String ciphertext= null;

      if (plaintext!= null) {
         try {
            Encryptor encryptor = JavaEncryptor.getInstance();
            CipherText ct = encryptor.encrypt(key, new PlaintText(plaintext));
            ciphertext = Base64.encodeBytes(ct.asPortableSerializedByteArray());
         } catch (EncryptionException e) {
            // handle exception. Perhaps set value to null?
         }
      }

      return ciphertext;
   }
}

Final Thoughts

There is no reason why you must have a one-to-one relationship between unencrypted and encrypted fields. It is perfectly reasonable to bundle related fields into a single value – in fact it is probably preferable to encrypting each field individually. The values could be represented in CSV, XML, JSON, or even a properties file.