My little project has come to a point where I need to keep sensitive information in a database. There are two approaches to protecting this information given if you have a “need to know” what the information is, or just need to compare it later on (verifying credentials). With passwords, you often store a hashed value along with a salt stored with the users account, and a pepper stored outside of the database such as an environment variable or configuration file. If you need access to the value for later, then we use encryption.

Secrets of Secrets

In either scenario, we need to assume a hacker will gain access to the database, and plan for protecting that information from being decrypted or compromised through other means. We need to not only keep secrets, but we also need to protect the sensitive information that lets us access those secrets. For starters, the decryption key(s) must not be stored in the same database. Where can these keys be stored?

• Hard-Coded
• Environment Variables
• File system
• Another database
• Certificate Manager
• Secret Manager

Hard-Coded

I’ve seen some systems where credentials are hard-coded in a PHP file. Although this is the simplest solution, it exposes your values if you find yourself storing the code into a source code repository. In addition, I’ve seen some systems where the same credentials are stored in multiple files, making it difficult to manage changes. Ideally you would want a single source of truth. The first fix to this problem would be to create a config.php file that populates variables that all other files would request. Just make sure to exclude this file from your source code repository via .gitignore and create/populate the file during your build process.

Environment Variables

Environment variables are one of the simplest and easiest ways to configure an application. On a server running Apache, we can define environment variables in an .htaccess file via SetEnv directives using the mod_env module. These environment variables are then exposed to PHP and accessed via getenv("NAME"). With node, many developers use dotenv files. Personally, this is the quickest way that I can get up and running with configurations.

Great care needs to be taken so that sensitive information in these files are not accidentally checked into the repository. In addition, these environment variables are accessible to anyone who can control what is in the PHP code. There is also risk that the variables could be inadvertently logged by system processes, especially if debugging a problem with verbose output. Any third party package that we install via composer has access to the same environment variables and could potentially leak our variables.

You could encrypt the environment values – but you still need to have access to the decrypted key that was used to encrypt the values. We are in an egg and chicken scenario.

So question – should my CI environment even be aware of these values? They are not used to do the actual build, only to populate the .htaccess file based on if I’m deploying to development or production. I have a desire to move the sensitive information out of the CI environment that isn’t necessary to build/deploy the project.

File System

Although the .htaccess file is technically on the the file system – it’s still accessible if the build servers FTP Account for the given domain was compromised. A .htaccess file in the root directory would be the first file that a hacker would download since it may reveal credentials, api keys, etc. Many of my hosts in the past have a root folder outside of what is publicly accessible to the websites. From here, I can create a subfolder that contains sensitive information such as a configuration JSON file, or certificate PEM files. Not even the build servers could access these files with their FTP account. Unfortunately, this also means that the secrets couldn’t be managed through the build server without some kind of endpoint that the build server could access to change configuration information during the build process – which would lead to potential security holes if not done correctly.

Another Database

We could potentially create a database whose sole purpose is to hold decryption keys. It would be a much smaller database in size. We have an unlimited number of databases. It would only make sense that our credentials to access this database is different than all other databases as well. This would add an overhead to connect to it unless we could cache the data somehow. It would definitely be easier to rotate the keys over time.

Certificate / Key Manager

If we are only storing encryption keys, we may be able to use openssl to store and retrieve certificates. I’m unfamiliar with it, but it looks like Libsodium (Sodium) may have a key management system.

Secret Manager

There are third party systems available that can manage secrets

• HashiCorp Vault
• Akeyless
• AWS Secrets Manager
• Google Cloud Secret Manager

A third party management solution would introduce the overhead of network requests to retrieve the secrets. Caching would be ideal in this situation. A third party introduces additional costs and having to handle outages when the service is not available from your host. If the costs are prohibitive, it may be cheaper to setup a secret manager on another web host under our control that we could call to retrieve secrets.

Moving Configuration

At this time, I wish to move sensitive information out of my .htaccess file and into a folder that is inaccessible to the web and the build servers FTP account. In addition, I would like to encrypt that file – or its values. This means that an encryption key will still need to be stored in .htaccess, or in a file that can still be read separately.

Let’s start off with a small test. Can we require a file outside of the /public_html/ folder and process it?

And… yes. I was afraid that I may not have permission to access files outside of public_html. I can include a PHP file outside of the public_html folder.

Should I encrypted the entire file, or just the values? From a code management perspective, its going to be easy to update the file if only the values are encrypted. How do I want to encrypt the data? This is going to be called with every request. What is fast? Symmetric encryption is much faster than Asymmetric encryption. It may be better to go with AES. OpenSSL has plenty built in for us to make use of. We need to do the following:

• Choose a key length (256 bit?)
• Create an encryption key
• Store the encryption key
• Read the encryption key
• Encrypt Data in key/value pair
• Update Existing Data for key
• Decrypt Data for given key
• Rotate Key
  • Decrypt all data with old key
  • Encrypt all data with new key
• Keep an archive of old keys
• Protect management of keys
• Automate key rotation via cron

That seems like the basics. If the key is compromised, we can build in an automated solution to rotate the key. Keeping an archive of the old keys seems fairly important since we don’t want to be locked out of a system if the only credentials we have is encrypted – that information could potentially be lost. I may need to address what to do with archived encryption keys and data to store them somewhere else.

I’m going to settle on a key/value store saved as JSON. I’ll encrypt only the values rather than the entire file, so the PHP script will have less overhead needed to process only what it needs.

Rotation

The first part to tackle is rotating the keys. Load the key and data files if they exist. Create a new key. Loop through existing keys, decrypt them with the old key, and encrypt them with the new key. Save the file. As an added measure, backup the old files.

<?php
require_once "./.common.php";

function main()
{
    global $keyFilePath;
    global $dataFilePath;
    global $BIT_LENGTH;
    global $keyFile;
    global $dataFile;

    $oldKeyPath = "$keyFilePath/$keyFile";
    $oldDataPath = "$dataFilePath/$dataFile";

    $old_key = loadKey();
    $secrets = loadSecrets();

    $new_key = generateKey();
    $secrets = rotateKeys($secrets, $old_key, $new_key);

    $timestamp = time();

    $tempKeyPath = "$keyFilePath/$timestamp.$keyFile";
    $tempDataPath = "$dataFilePath/$timestamp.$dataFile";

    ensureDirExists($keyFilePath);
    ensureDirExists($dataFilePath);

    saveAsPEM($tempKeyPath, $new_key);
    saveAsJson($tempDataPath, $secrets);

    if (file_exists($oldKeyPath)) {
        copyFile($oldKeyPath, "$keyFilePath/$timestamp.$keyFile.old");
    }

    if (file_exists($oldDataPath)) {
        copyFile($oldDataPath, "$dataFilePath/$timestamp.$dataFile.old");
    }

    copyFile($tempKeyPath, $oldKeyPath);
    copyFile($tempDataPath, $oldDataPath);
}

function generateKey()
{
    global $BIT_LENGTH;
    return openssl_random_pseudo_bytes($BIT_LENGTH / 8);
}

function rotateKeys($data, $old_key, $new_key)
{
    global $BIT_LENGTH;
    if (empty($data)) {
        return [];
    }
    foreach ($data as $name => $encrypted) {
        $clear = decrypt($encrypted, $old_key);
        $data[$name] = encrypt($clear, $new_key);
    }
    return $data;
}

try {
    main();
    echo "Completed.";
} catch (Exception $e) {
    haltExecution("Unexpected exception.", $e->getMessage());
}

I proceeded to setup a cron job to rotate the keys once a month. I’d prefer to do it once every 60 or 90 days, but the interface on hostinger doesn’t allow me to fine-tune the cron job settings or provide my own file. I’m uncertain if they permit me to setup cron jobs via ssh since they already provide a web interface to manage it.

Set Secrets

Now that we can rotate the keys, we need some values to be encrypted. On top of that, we need to restrict access by a password so that only authorized users may add and change keys. If the secrets do not yet have a password, we can use the password provided to create a new entry.

<?php
require_once "./.common.php";

function main()
{
    if ($_SERVER["REQUEST_METHOD"] !== "POST") {
        return;
    }

    $name = $_POST["name"];
    $value = $_POST["value"];
    $password = $_POST["password"];

    $secrets = openSecrets($password);
    $key = loadKey();
    $secrets[$name] = encrypt($value, $key);
    saveSecrets($secrets);
}

try {
    main();
} catch (Exception $e) {
    haltExecution("Unexpected exception.", $e->getMessage());
}

?>
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Set Secret</title>
</head>
<body>
    <form method="post" action="<?php echo htmlspecialchars($_SERVER["PHP_SELF"]); ?>">
        Name: <input type="text" name="name"><br><br>
        Value: <input type="text" name="value"><br><br>
        Password: <input type="password" name="password"><br><br>
        <input type="submit" value="Submit">
    </form>
    <em>If the secrets does not yet exist, the provided password will be saved within it as <?php echo $uiPasswordName ?>.</em>
</body>
</html>

Reveal Secrets

Last, let’s create a page that reveals all of the secrets. Just like our page to set secrets, we can save the provided password to the secrets file if it doesn’t already have an encrypted password to protect it. And for added measure, we can add the option to show only the names of our keys, or both the names and values.

<?php
require_once "./.common.php";

function main()
{
    if ($_SERVER["REQUEST_METHOD"] !== "POST") {
        return;
    }

    $password = $_POST["password"];
    $show = $_POST["show"];

    $secrets = openSecrets($password);
    $key = loadKey();

    echo '<table border="1"><tr><td>Name</td><td>Value</td></tr>';
    foreach ($secrets as $name => $encrypted) {
        if ($show === "key-pairs") {
            $clear = decrypt($encrypted, $key);
        } else {
            $clear = "****";
        }
        echo "<tr><td>$name</td><td>$clear</td></tr>";
    }
    echo '</table>';
}

try {
    main();
} catch (Exception $e) {
    haltExecution("Unexpected exception.", $e->getMessage());
}

?>
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Reveal</title>
</head>
<body>
    <form method="post" action="<?php echo htmlspecialchars($_SERVER["PHP_SELF"]); ?>">
      <input type="radio" name="show" value="keys">Names Only<br>
      <input type="radio" name="show" value="key-pairs">Names & Values<br>
      <input type="password" name="password">
      <input type="submit" value="Submit">
    </form>
    <em>If the secrets does not yet exist, the provided password will be saved within it as <?php echo $uiPasswordName ?>.</em>
</body>
</html>

It’s not a friendly interface, but it gets the job done. The existing secret can be changed on the set page by entering secrets_password as the name, the new password as the value, and the old password before submitting.

So… what’s in .common.php? That’s where all the magic happens.

<?php
$keyFilePath = getEnv("SECRETS_KEY_PATH");
$dataFilePath = getEnv("SECRETS_DATA_PATH");
$isProduction = getEnv("ENVIRONMENT") === "PRODUCTION";
$BIT_LENGTH = 256;
$keyFile = "aes-$BIT_LENGTH-key.pem";
$dataFile = 'data.json';
$uiPasswordName = 'secrets_password';

function openSecrets($password)
{
    global $uiPasswordName;

    if (empty($password)) {
        haltExecution("Empty password");
    } elseif (strlen($password) < 8) {
        haltExecution("Short password");
    }

    $key = loadKey();
    $secrets = loadSecrets();
    if (array_key_exists($uiPasswordName, $secrets)) {
        $expectedPassword = decrypt($secrets[$uiPasswordName], $key);
        if ($password !== $expectedPassword) {
            haltExecution("Wrong password");
        }
    } else {
        $secrets[$uiPasswordName] = encrypt($password, $key);
        saveSecrets($secrets);
    }
    return $secrets;
}

function loadKey()
{
    global $keyFilePath;
    global $keyFile;

    $path = "$keyFilePath/$keyFile";
    $key = loadIfExists($path);
    if (strpos($key, "-----BEGIN AES KEY-----") === false) {
        return $key;
    }
    $key = getPemKey($key);
    return $key;
}
function loadSecrets()
{
    global $dataFilePath;
    global $dataFile;

    $path = "$dataFilePath/$dataFile";
    return loadJsonIfExists($path);
}
function saveSecrets($secrets)
{
    global $dataFilePath;
    global $dataFile;
    $path = "$dataFilePath/$dataFile";
    saveAsJson($path, $secrets);
}
function loadIfExists($file)
{
    if (file_exists($file)) {
        $contents = file_get_contents($file);
        if ($contents === false) {
            haltExecution("Error reading '$file'.");
        }
        return $contents;
    }
}

function loadJsonIfExists($file)
{
    $json = loadIfExists($file);
    if (!empty($json)) {
        $array = json_decode($json, true);
        if ($array === null) {
            haltExecution("JSON decoding error.", json_last_error_msg());
        }
        return $array;
    }
}

function haltExecution($message, $additionalMessage = '')
{
    global $isProduction;
    if ($isProduction) {
        echo "An error occurred";
    } else {
        echo $message;
        if (!empty($additionalMessage)) {
            echo ": $additionalMessage";
        }
        $lastError = error_get_last();
        if ($lastError !== null) {
            $errorMessage = $lastError['message'];
            echo ": $errorMessage";
        }
    }
    exit();
}

function getPemKey($pem)
{
    $beginMarker = "-----BEGIN AES KEY-----";
    $endMarker = "-----END AES KEY-----";
    $startPos = strpos($pem, $beginMarker);
    $endPos = strpos($pem, $endMarker);
    if ($startPos === false || $endPos === false) {
        return false;
    }
    $startPos += strlen($beginMarker);
    $keyData = substr($pem, $startPos, $endPos - $startPos);
    return base64_decode($keyData);
}
function checkKey($key)
{
    global $BIT_LENGTH;
    if (empty($key)) {
        haltExecution("Missing encryption key.");
    }
    $actual_bits = strlen($key) * 8;

    if ($actual_bits === $BIT_LENGTH) {
        return;
    }

    $parsedKey = getPemKey($key);
    if ($parsedKey === false) {
        haltExecution("$actual_bits bit key expected to be $BIT_LENGTH bit.");
    }

    $actual_bits = strlen($parsedKey) * 8;
    if ($actual_bits !== $BIT_LENGTH) {
        haltExecution("$actual_bits bit key expected to be $BIT_LENGTH bit.");
    }
}
function save($file, $data)
{
    $result = file_put_contents($file, $data);

    if ($result === false) {
        haltExecution("Unable to write to $file.");
    }
}

function saveAsJson($file, $data)
{
    $json = json_encode($data);

    if ($json === false) {
        haltExecution('Error encoding data to JSON', json_last_error_msg());
    }
    save($file, $json);
}

function ensureDirExists($path)
{
    if (!is_dir($path)) {
        // all for owner, read/execute for others
        $permissions = 0755;
        $result = mkdir($path, $permissions, true);
        if ($result === false) {
            haltExecution("Unable to make directory for $path.");
        }
    }
}

function saveAsPEM($file, $key)
{
    $beginMarker = "-----BEGIN AES KEY-----";
    $endMarker = "-----END AES KEY-----";
    $encoded = base64_encode($key);
    $pem = "$beginMarker\n$encoded\n$endMarker\n";
    save($file, $pem);
}

function copyFile($old_name, $new_name)
{
    $result = copy($old_name, $new_name);
    if ($result === false) {
        haltExecution("Unable to copy '$old_name' to '$new_name'");
        exit;
    }
}

function decrypt($encryptedValue, $key)
{
    global $BIT_LENGTH;
    checkKey($key);

    $method = "aes-$BIT_LENGTH-cbc";
    $iv_length = openssl_cipher_iv_length($method);
    $iv = substr(base64_decode($encryptedValue), 0, $iv_length);
    $encrypted = substr(base64_decode($encryptedValue), $iv_length);
    $decrypted = openssl_decrypt($encrypted, $method, $key, OPENSSL_RAW_DATA, $iv);
    if ($decrypted === false) {
        haltExecution("Unable to decrypt.", openssl_error_string());
    }
    return $decrypted;
}

function encrypt($value, $key)
{
    global $BIT_LENGTH;
    checkKey($key);

    $method = "aes-$BIT_LENGTH-cbc";
    $iv_length = openssl_cipher_iv_length($method);
    $iv = openssl_random_pseudo_bytes($iv_length);
    $encrypted = openssl_encrypt($value, $method, $key, OPENSSL_RAW_DATA, $iv);
    if ($encrypted === false) {
        haltExecution("Unable to encrypt.", openssl_error_string());
    }
    return base64_encode($iv . $encrypted);
}

All together, these files make up a rudimentary secrets manager. The environment variables are still used – but now they are only used to point us to where the secrets file and key are located at. Everything else can move out of .htaccess and be entered into our secrets manager.

The manager stores everything in my secrets folder. As keys are rotated, backups are made with the timestamps.

Is this better? In terms of security, I believe so. I haven’t setup any FTP accounts to access the root folder that contains public_html. The file is not accessible over HTTP. All values are encrypted along with iv salt values. Keys are rotated monthly. The secrets can not be modified unless you provide a password.

In terms of speed, I’m uncertain of the impact. We need to decrypt our database credentials and a few other settings. This involves reading two files and calling openssl a few times to decrypt what we need. I’m certain there is a performance penalty, but I am uncertain what the ramifications are.

What could be improved?

• Access Control – Unable to restrict keys by scope or users. Various parts of the application don’t need access to everything.
  • Maybe use a different key for different values
• Replication. Unable to synchronize values between multiple servers.
• Auditing. Who changed a key value? Who accessed it? When?
• Deleting keys. There isn’t a way to delete keys, but it would be fairly simple to setup.
• Versioning. I don’t save the original values before updating a key. Once a keys value is changed, the original value is lost.
• The key file should be stored somewhere else instead of the same folder as the secrets data.
• Caching – can the encrypted values be decrypted and stored in memory somewhere to reduce the overhead of reading files and decrypting data?
• Clunky interface.
• Intrusion detection / preventing DOS attacks
• IP restriction
• Automate changing/adding secrets via build server