import { dynamicStringHandler } from "@jjHelpers/dynamicStringHandler"; import { getJJConfigByKeyChain } from "@jjHelpers/jjConfigUtils"; import { makeJJlogger } from "@jjHelpers/jjLogger"; import _ from "lodash"; import fs from "fs-extra"; import path from "path"; import { generateHasDir, generateMD5, jjDecrypt, jjEncrypt } from "@jjHelpers/cryptoUtils"; import { saveFileIfItChanged } from "@jjHelpers/saveFileIfItChanged"; import * as YAML from "js-yaml"; import { subRepositoryClean, subRepositoryPull, subRepositoryPushChanges } from "@jjHelpers/gitSubRepoUtils"; import moment from "moment"; import { bashRunAndReturnSync } from "@jjHelpers/bashUtils"; const logger = makeJJlogger({ filename: "jj/cli/secrets.ts", }); export const getSecretFolder = () => { let secretsFolder = getJJConfigByKeyChain(`secretsFolder`); secretsFolder = dynamicStringHandler(secretsFolder, "", process.env, false); return secretsFolder; }; export const getEncryptionKeyForEnv = (envKey) => { const secretsFolder = getSecretFolder(); if (!fs.existsSync(`${secretsFolder}/~~${envKey}`)) { logger.error(`Encryption key for ${envKey} env not fond and nt exist `); process.exit(1); } const encryptionKey = fs.readFileSync(`${secretsFolder}/~~${envKey}`, "utf8"); return encryptionKey; }; export const decryptFile = async (absFilePath) => { if (!fs.existsSync(absFilePath)) { logger.error(`Secret ${absFilePath} is not exist`); } logger.log(`decrypt file-secret for ${absFilePath} start`); const secretFileObjRaw = await fs.readFile(absFilePath, "utf8"); const secretObj = JSON.parse(secretFileObjRaw); const envKey = path.basename(path.join(absFilePath, "..")); const decryptedSecretObj = {}; const encryptionKey = getEncryptionKeyForEnv(envKey); Object.keys(secretObj).forEach((key) => { const plainValue = secretObj[key]; if (plainValue.includes(`${envKey}--`)) { decryptedSecretObj[key] = jjDecrypt(plainValue.replace(`${envKey}--`, ""), encryptionKey); return; } decryptedSecretObj[key] = plainValue; return; }); let shFormatDecrypt = ""; Object.keys(decryptedSecretObj).forEach((key) => { shFormatDecrypt += `export ${key}="${decryptedSecretObj[key]}"\n`; }); const absDirPath = path.dirname(absFilePath); const fileName = path.basename(absFilePath).replace(".json", ""); if (!fs.existsSync(`${absDirPath}/~~decrypt`)) { fs.mkdirSync(`${absDirPath}/~~decrypt`, { recursive: true }); } const decryptedJsonFormat = `${absDirPath}/~~decrypt/${fileName}.json`; fs.writeFileSync(decryptedJsonFormat, JSON.stringify(decryptedSecretObj, null, 4), "utf-8"); const decryptedShFormat = `${absDirPath}/~~decrypt/${fileName}.sh`; fs.writeFileSync(decryptedShFormat, shFormatDecrypt, "utf-8"); logger.success(`decrypt file-secret for ${absFilePath} done`); return decryptedSecretObj; }; export const setSecretFileFromFile = async ({ absFilePath, secretKey, secretAbsFilePath }) => { if (!fs.existsSync(secretAbsFilePath)) { logger.error(`Secret original ${secretAbsFilePath} is not exist`); } const secretValue = fs.readFileSync(secretAbsFilePath, "utf8"); return await setSecretFile({ absFilePath, secretKey, secretValue, }); }; export const setSecretFile = async ({ absFilePath, secretKey, secretValue }) => { if (!fs.existsSync(absFilePath)) { logger.error(`Secret ${absFilePath} is not exist`); } logger.log(`adding secret for ${absFilePath} with Key ${secretKey.toUpperCase()} start`); const envKey = path.basename(path.join(absFilePath, "..")); const secretFileObjRaw = await fs.readFile(absFilePath, "utf8"); const secretObj = JSON.parse(secretFileObjRaw); const encryptionKey = getEncryptionKeyForEnv(envKey); const encryptVal = jjEncrypt(secretValue, encryptionKey); secretObj[secretKey.toUpperCase()] = `${envKey}--${encryptVal}`; saveFileIfItChanged(JSON.stringify(secretObj, null, 4), absFilePath); logger.success(`adding secret for ${absFilePath} with Key ${secretKey.toUpperCase()} done`); await decryptFile(absFilePath); }; export const setRawFile = async ({ absFilePath, secretKey, secretValue }) => { if (!fs.existsSync(absFilePath)) { logger.error(`Secret ${absFilePath} is not exist`); } logger.log(`adding raw secret for ${absFilePath} with Key ${secretKey.toUpperCase()} start`); const secretFileObjRaw = await fs.readFile(absFilePath, "utf8"); const secretObj = JSON.parse(secretFileObjRaw); secretObj[secretKey.toUpperCase()] = secretValue; saveFileIfItChanged(JSON.stringify(secretObj, null, 4), absFilePath); logger.success(`adding secret for ${absFilePath} with Key ${secretKey.toUpperCase()} done`); await decryptFile(absFilePath); }; export const commitSecretMeta = async () => { const metadataRepoFolderRelative = getJJConfigByKeyChain(`metadataRepo.path`); const metadataRepoBranch = getJJConfigByKeyChain(`metadataRepo.branch`); await subRepositoryClean(metadataRepoFolderRelative); await subRepositoryPull(metadataRepoFolderRelative, metadataRepoBranch); logger.log(`commitSecretMeta for ${process.env.JJ_CLI_SELECTED_COMMIT} start`); const environments = getJJConfigByKeyChain(`environments`); const environmentsKeys = Object.keys(environments); const metadataSecretPathBase = `${process.env.JJ_WORKSPACE_DIR}/${metadataRepoFolderRelative}/metadata/${process.env.JJ_CLI_SELECTED_COMMIT}`; if (!fs.existsSync(metadataSecretPathBase)) { fs.mkdirSync(metadataSecretPathBase, { recursive: true }); } const metadataSecretPathBaseCommitSecrets = `${metadataSecretPathBase}/jj_secrets.yaml`; if (!fs.existsSync(metadataSecretPathBaseCommitSecrets)) { await fs.writeFile(metadataSecretPathBaseCommitSecrets, YAML.dump({}, {}), "utf8"); } const metadataSecretPathBaseCommitSecretsObj = YAML.load(await fs.readFile(metadataSecretPathBaseCommitSecrets, "utf8")) || {}; const secretsBasePath = `${process.env.JJ_WORKSPACE_DIR}/${metadataRepoFolderRelative}/secretsHash`; const mainSecretMetaDataFilePath = `${process.env.JJ_WORKSPACE_DIR}/${metadataRepoFolderRelative}/secrets.yaml`; if (!fs.existsSync(mainSecretMetaDataFilePath)) { await fs.writeFile(mainSecretMetaDataFilePath, YAML.dump({}, {}), "utf8"); } const mainSecretMetaDataObj = YAML.load(await fs.readFile(mainSecretMetaDataFilePath, "utf8")) || {}; for (const environmentKey of environmentsKeys) { const envMetadataFolder = `${metadataSecretPathBase}/${environmentKey}`; if (!fs.existsSync(envMetadataFolder)) { fs.mkdirSync(envMetadataFolder, { recursive: true }); } const secretsMetadataPath = `${envMetadataFolder}/jj_secrets.yaml`; if (!fs.existsSync(secretsMetadataPath)) { await fs.writeFile(secretsMetadataPath, YAML.dump({}, {}), "utf8"); } const currentMetadataSecretsObj = YAML.load(await fs.readFile(secretsMetadataPath, "utf8")) || {}; const secretsHashEnvPath = `${secretsBasePath}/${environmentKey}`; if (!fs.existsSync(secretsHashEnvPath)) { fs.mkdirSync(secretsHashEnvPath, { recursive: true }); } const currentSecretsEnvPath = `${process.env.JJ_WORKSPACE_DIR}/secrets/${environmentKey}`; let jsonSourceFormatEncryptedSecrets = fs.readdirSync(currentSecretsEnvPath); jsonSourceFormatEncryptedSecrets = jsonSourceFormatEncryptedSecrets.filter((file) => file.includes(".json")); for (const jsonSourceFormatEncryptedSecret of jsonSourceFormatEncryptedSecrets) { const encryptedSecretFilePath = `${currentSecretsEnvPath}/${jsonSourceFormatEncryptedSecret}`; const encryptedSecretFileHash = (await generateHasDir(encryptedSecretFilePath)).hash; const secretFileNameWithoutExtension = jsonSourceFormatEncryptedSecret.split(".json")[0]; const secretHashFileNameWithoutExtension = `${secretFileNameWithoutExtension}__${encryptedSecretFileHash}`; const secretHashFileName = `${secretHashFileNameWithoutExtension}.json`; const secretsHashFile = `${secretsHashEnvPath}/${secretHashFileName}`; if (!fs.existsSync(secretsHashFile)) { fs.copyFileSync(encryptedSecretFilePath, secretsHashFile); } _.set(currentMetadataSecretsObj, `secrets.${environmentKey}.${secretFileNameWithoutExtension}`, encryptedSecretFileHash); const currentSecretMdObj = _.get( mainSecretMetaDataObj, `['${secretFileNameWithoutExtension}']['${environmentKey}']['${encryptedSecretFileHash}']`, {} ); _.set(mainSecretMetaDataObj, `['${secretFileNameWithoutExtension}']['${environmentKey}']['${encryptedSecretFileHash}']`, { ...currentSecretMdObj, timeOfInitiate: currentSecretMdObj?.timeOfInitiate ? currentSecretMdObj?.timeOfInitiate : moment().toISOString(), firstCommit: currentSecretMdObj?.firstCommit ? currentSecretMdObj?.firstCommit : process.env.JJ_CLI_SELECTED_COMMIT, }); await fs.writeFile(secretsMetadataPath, YAML.dump(currentMetadataSecretsObj, {}), "utf8"); _.merge(metadataSecretPathBaseCommitSecretsObj, currentMetadataSecretsObj); } await fs.writeFile(mainSecretMetaDataFilePath, YAML.dump(mainSecretMetaDataObj, {}), "utf8"); } await fs.writeFile(metadataSecretPathBaseCommitSecrets, YAML.dump(metadataSecretPathBaseCommitSecretsObj, {}), "utf8"); await subRepositoryPushChanges( metadataRepoFolderRelative, metadataRepoBranch, `${process.env.JJ_CLI_SELECTED_COMMIT} ${process.env.JJ_GIT_USER_EMAIL} 'jj secrets commitSecretMeta'` ); logger.success(`commitSecretMeta for ${process.env.JJ_CLI_SELECTED_COMMIT} done`); // try { // const res = await bashRunAndReturnSync({ // command: `jj portainer syncSecret`, // }); // console.log(res); // } catch (err) { // // // } // }; export const secrets = async () => { if (process.env.JJ_CLI_ARG_2 == "commitSecretMeta") { await commitSecretMeta(); process.exit(0); } const secretFilePath = process.env.JJ_CLI_ARG_3; const absFilePath = path.resolve(process.cwd(), secretFilePath); const envKey = path.basename(path.join(absFilePath, "..")); const secretKey = process.env.JJ_CLI_ARG_4; const secretValue = process.env.JJ_CLI_ARG_5; const environment = getJJConfigByKeyChain(`environments['${envKey}']`); if (_.isEmpty(environment)) { logger.error(`environment is not exist`); process.exit(1); } if (process.env.JJ_CLI_ARG_2 == "setRaw") { await setRawFile({ absFilePath, secretKey, secretValue, }); process.exit(0); } if (process.env.JJ_CLI_ARG_2 == "setSecret") { await setSecretFile({ absFilePath, secretKey, secretValue, }); process.exit(0); } if (process.env.JJ_CLI_ARG_2 == "decryptFile") { await decryptFile(absFilePath); process.exit(0); } if (process.env.JJ_CLI_ARG_2 == "setSecretFromFile") { await setSecretFileFromFile({ absFilePath, secretKey, secretAbsFilePath: secretValue, }); process.exit(0); } }; export default secrets;