import { dynamicStringHandler } from "@jjHelpers/dynamicStringHandler"; import { getJJConfigByKeyChain } from "@jjHelpers/jjConfigUtils" import { makeJJlogger } from "@jjHelpers/jjLogger"; import _ from 'lodash'; import fs from 'fs-extra'; import path from "path"; import { generateHasDir, generateMD5, jjDecrypt, jjEncrypt } from "@jjHelpers/cryptoUtils"; import { saveFileIfItChanged } from "@jjHelpers/saveFileIfItChanged"; import * as YAML from "js-yaml"; import { subRepositoryClean, subRepositoryPull, subRepositoryPushChanges } from "@jjHelpers/gitSubRepoUtils"; import moment from "moment"; const logger = makeJJlogger({ filename: 'jj/cli/secrets.ts' }) const getSecretFolder = () => { let secretsFolder = getJJConfigByKeyChain(`secretsFolder`) secretsFolder = dynamicStringHandler(secretsFolder, "", process.env, false) return secretsFolder } const getEncryptionKeyForEnv = (envKey) => { const secretsFolder = getSecretFolder() if (!fs.existsSync(`${secretsFolder}/~~${envKey}`)) { logger.error(`Encryption key for ${envKey} env not fond and nt exist `) process.exit(1) } const encryptionKey = fs.readFileSync(`${secretsFolder}/~~${envKey}`, 'utf8') return encryptionKey } const decryptFile = async (absFilePath) => { if (!fs.existsSync(absFilePath)) { logger.error(`Secret ${absFilePath} is not exist`) } logger.log(`decrypt file-secret for ${absFilePath} start`) const secretFileObjRaw = await fs.readFile(absFilePath, 'utf8') const secretObj = JSON.parse(secretFileObjRaw) const envKey = path.basename(path.join(absFilePath, "..")) const decryptedSecretObj = {} const encryptionKey = getEncryptionKeyForEnv(envKey) Object.keys(secretObj).forEach((key) => { const plainValue = secretObj[key] if (plainValue.includes(`${envKey}--`)) { decryptedSecretObj[key] = jjDecrypt(plainValue.replace(`${envKey}--`, ""), encryptionKey) return } decryptedSecretObj[key] = plainValue return }) let shFormatDecrypt = "" Object.keys(decryptedSecretObj).forEach((key) => { shFormatDecrypt += `export ${key}="${decryptedSecretObj[key]}"\n` }) const absDirPath = path.dirname(absFilePath); const fileName = path.basename(absFilePath).replace('.json', ''); if (!fs.existsSync(`${absDirPath}/~~decrypt`)) { fs.mkdirSync(`${absDirPath}/~~decrypt`, { recursive: true }) } const decryptedJsonFormat = `${absDirPath}/~~decrypt/${fileName}.json` fs.writeFileSync(decryptedJsonFormat, JSON.stringify(decryptedSecretObj, null, 4), 'utf-8'); const decryptedShFormat = `${absDirPath}/~~decrypt/${fileName}.sh` fs.writeFileSync(decryptedShFormat, shFormatDecrypt, 'utf-8'); logger.success(`decrypt file-secret for ${absFilePath} done`) } const setSecretFileFromFile = async ({ absFilePath, secretKey, secretAbsFilePath }) => { if (!fs.existsSync(secretAbsFilePath)) { logger.error(`Secret original ${secretAbsFilePath} is not exist`) } const secretValue = fs.readFileSync(secretAbsFilePath, 'utf8') return await setSecretFile({ absFilePath, secretKey, secretValue }) } const setSecretFile = async ({ absFilePath, secretKey, secretValue }) => { if (!fs.existsSync(absFilePath)) { logger.error(`Secret ${absFilePath} is not exist`) } logger.log(`adding secret for ${absFilePath} with Key ${secretKey.toUpperCase()} start`) const envKey = path.basename(path.join(absFilePath, "..")) const secretFileObjRaw = await fs.readFile(absFilePath, 'utf8') const secretObj = JSON.parse(secretFileObjRaw) const encryptionKey = getEncryptionKeyForEnv(envKey) const encryptVal = jjEncrypt(secretValue, encryptionKey) secretObj[secretKey.toUpperCase()] = `${envKey}--${encryptVal}` saveFileIfItChanged(JSON.stringify(secretObj, null, 4), absFilePath) logger.success(`adding secret for ${absFilePath} with Key ${secretKey.toUpperCase()} done`) await decryptFile(absFilePath) } const commitSecretMeta = async () => { const metadataRepoFolderRelative = getJJConfigByKeyChain(`metadataRepo.path`) const metadataRepoBranch = getJJConfigByKeyChain(`metadataRepo.branch`) await subRepositoryClean(metadataRepoFolderRelative) await subRepositoryPull(metadataRepoFolderRelative, metadataRepoBranch) logger.log(`commitSecretMeta for ${process.env.JJ_CLI_SELECTED_COMMIT} start`) const environments = getJJConfigByKeyChain(`environments`) const environmentsKeys = Object.keys(environments) const metadataSecretPathBase = `${process.env.JJ_WORKSPACE_DIR}/${metadataRepoFolderRelative}/metadata/${process.env.JJ_CLI_SELECTED_COMMIT}` const secretsBasePath = `${process.env.JJ_WORKSPACE_DIR}/${metadataRepoFolderRelative}/secretsHash` const mainSecretMetaDataFilePath = `${process.env.JJ_WORKSPACE_DIR}/${metadataRepoFolderRelative}/secrets.yaml` if (!fs.existsSync(mainSecretMetaDataFilePath)) { await fs.writeFile(mainSecretMetaDataFilePath, YAML.dump({}, {}), "utf8") } const mainSecretMetaDataObj = YAML.load(await fs.readFile(mainSecretMetaDataFilePath, "utf8")) || {} for (const environmentKey of environmentsKeys) { const envMetadataFolder = `${metadataSecretPathBase}/${environmentKey}` if (!fs.existsSync(envMetadataFolder)) { fs.mkdirSync(envMetadataFolder, { recursive: true }) } const secretsMetadataPath = `${envMetadataFolder}/jj_secrets.yaml` if (!fs.existsSync(secretsMetadataPath)) { await fs.writeFile(secretsMetadataPath, YAML.dump({}, {}), "utf8") } const currentMetadataSecretsObj = YAML.load(await fs.readFile(secretsMetadataPath, "utf8")) || {} const secretsHashEnvPath = `${secretsBasePath}/${environmentKey}` if (!fs.existsSync(secretsHashEnvPath)) { fs.mkdirSync(secretsHashEnvPath, { recursive: true }) } const currentSecretsEnvPath = `${process.env.JJ_WORKSPACE_DIR}/secrets/${environmentKey}` let jsonSourceFormatEncryptedSecrets = fs.readdirSync(currentSecretsEnvPath) jsonSourceFormatEncryptedSecrets = jsonSourceFormatEncryptedSecrets.filter((file) => file.includes(".json")) for (const jsonSourceFormatEncryptedSecret of jsonSourceFormatEncryptedSecrets) { const encryptedSecretFilePath = `${currentSecretsEnvPath}/${jsonSourceFormatEncryptedSecret}` const encryptedSecretFileHash = (await generateHasDir(encryptedSecretFilePath)).hash const secretFileNameWithoutExtension = jsonSourceFormatEncryptedSecret.split(".json")[0] const secretHashFileNameWithoutExtension = `${secretFileNameWithoutExtension}__${encryptedSecretFileHash}` const secretHashFileName = `${secretHashFileNameWithoutExtension}.json` const secretsHashFile = `${secretsHashEnvPath}/${secretHashFileName}` if (!fs.existsSync(secretsHashFile)) { fs.copyFileSync(encryptedSecretFilePath, secretsHashFile) } _.set(currentMetadataSecretsObj, `secrets.${environmentKey}.${secretFileNameWithoutExtension}`, encryptedSecretFileHash) const currentSecretMdObj = _.get(mainSecretMetaDataObj, `['${secretFileNameWithoutExtension}']['${environmentKey}']['${encryptedSecretFileHash}']`, {}) _.set( mainSecretMetaDataObj, `['${secretFileNameWithoutExtension}']['${environmentKey}']['${encryptedSecretFileHash}']`, { ...currentSecretMdObj, timeOfInitiate: currentSecretMdObj?.timeOfInitiate ? currentSecretMdObj?.timeOfInitiate : moment().toISOString(), firstCommit: currentSecretMdObj?.firstCommit ? currentSecretMdObj?.firstCommit : process.env.JJ_CLI_SELECTED_COMMIT, } ) await fs.writeFile(secretsMetadataPath, YAML.dump(currentMetadataSecretsObj, {}), "utf8") } } await fs.writeFile(mainSecretMetaDataFilePath, YAML.dump(mainSecretMetaDataObj, {}), "utf8") await subRepositoryPushChanges(metadataRepoFolderRelative, metadataRepoBranch, `${process.env.JJ_CLI_SELECTED_COMMIT} ${process.env.JJ_GIT_USER_EMAIL} 'jj secrets commitSecretMeta'`) logger.success(`commitSecretMeta for ${process.env.JJ_CLI_SELECTED_COMMIT} done`) } const secrets = async () => { if (process.env.JJ_CLI_ARG_2 == 'commitSecretMeta') { await commitSecretMeta() process.exit(0) } const secretFilePath = process.env.JJ_CLI_ARG_3 const absFilePath = path.resolve(process.cwd(), secretFilePath) const envKey = path.basename(path.join(absFilePath, "..")) const secretKey = process.env.JJ_CLI_ARG_4 const secretValue = process.env.JJ_CLI_ARG_5 const environment = getJJConfigByKeyChain(`environments['${envKey}']`) if (_.isEmpty(environment)) { logger.error(`environment is not exist`) process.exit(1) } if (process.env.JJ_CLI_ARG_2 == 'setSecret') { await setSecretFile({ absFilePath, secretKey, secretValue }) process.exit(0) } if (process.env.JJ_CLI_ARG_2 == 'decryptFile') { await decryptFile(absFilePath) process.exit(0) } if (process.env.JJ_CLI_ARG_2 == 'setSecretFromFile') { await setSecretFileFromFile({ absFilePath, secretKey, secretAbsFilePath: secretValue }) process.exit(0) } } export default secrets