This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

CICD

Detailed Explanation of Cloudforet CICD Pipeline and Architecture

Actions

Cloudforet has 100+ repositories, 70 of which are for applications and these repositories have github action workflows.

Because of that, it's very difficult to handle one by one when you need to update or run a workflow.

To solve this problem, we created Actions.

The diagram below shows the relationship between Actions and repositories.

What does actions actually do?

Actions is a control tower that manages and deploys github action workflows for Cloudforet's core services.
It can also bulk trigger these workflows when a new version of Cloudforet's core services needs to be released.

1. Manage and deploy github action workflows for Cloudforet's core services.

All workflows for Cloudforet's core services are managed and deployed in this repository.

We write the workflow according to our workflow policy and put it in the workflows directory of Actions.
Then these workflows can be deployed into the repository of Cloudforet's core services

Our devops engineers can modify workflows according to our policy and deploy them in batches using this feature.

The diagram below shows the process for this feature.

*) If you want to see the Actions script that appears in the diagram, see here.

2. trigger workflows when a new version of Cloudforet's core services needs to be released.

When a new version of Cloudforet's core services is released, we need to trigger the workflow of each repository.
To do this, we made workflow that can trigger workflows of each repository in Actions.

Reference

Cloudforet CICD Project, Actions

1 - Frontend Microservice CI

Detailed Explanation of Frontend Microservice Repository CI

Frontend Microservice CI process details



The flowchart above describes 4 .yml GitHub Action files for CI process of frontend microservices. Unlike the backend microservices, frontend microservices are not released as packages, so the branch tagging job does not include building and uploading the NPM software package. Frontend microservices only build software and upload it on Docker, not NPM or PyPi.


To check the details, go to the .github/workflow directory in each directory. We provide an example of the workflow directory of the frontend microservices with the below link.


2 - Backend Microservice CI

Detailed Explanation of Backend Microservice Repository CI

Backend Microservice CI process details



The flowchart above describes 4 .yml GitHub Action files for CI process of backend microservices. Most of the workflow is similar to the frontend microservices' CI. However, unlike the frontend microservices, backend microservices are released as packages, therefore the process includes building and uploading PyPi package.


To check the details, go to the .github/workflow directory in each directory. We provide an example of the workflow directory of the backend microservices with the below link.


3 - Frontend Core Microservice CI

Detailed Explanation of Frontend Core Microservice Repository CI

Frontend Core Microservice CI



Frontend Core microservices' codes are integrated and built, uploaded with the flow explained above. Most of the workflows include set-up process including setting Node.js, caching node modules, and installing dependencies. After the set-up proccess, each repository workflow is headed to building process proceeded in NPM. After building, both repositories' packages built are released in NPM by code npm run semantic-release.


Check semantic-release site, npm: semantic-release for further details about the release process.


Also, unlike other repositories deployed by the flow from Docker to Spinnaker and k8s, spaceone-design-system repository is deployed differently, based on direct deployment through AWS S3.


To check the details, go to the .github/workflow directory in each directory.


4 - Backend Core Microservice CI

Detailed Explanation of Backend Core Microservice Repository CI

Backend Core Microservice CI process details



Backend Core microservices' 4 workflow related GitHub Action files are explained through the diagram above. Unlike the other repositories, pushes in GitHub with tags are monitored and trigger to do building the package in PyPi for testing purposes, instead of workflow tasks for master branch pushes.


Also, Backend Core microservices are not built and uploaded on Docker. They are only managed in PyPi.


To check the details, go to the .github/workflow directory in each directory.


5 - Plugin CI

Detailed Explanation of Plugin Repository CI

Plugin CI process details


Plugin repositories with name starting with ‘plugin-’ have unique CI process managed with workflow file named push_sync_ci.yaml. As the total architecture of CI is different from other repositories, plugin repositories' workflow files are automatically updated at every code commit.



We can follow the plugin CI process, step by step.


Step 1. push_sync_ci.yaml in each plugin repository is triggered by master branch push or in a manual way.

Step 2. push_sync_ci.yaml runs cloudforet-io/actions/.github/worflows/deploy.yaml.

Step 2-1. spaceone/actions/.github/worflows/deploy.yaml runs cloudforet-io/actions/src/main.py.

  1. cloudforet-io/actions/src/main.py updates each plugin repository workflow files based on the repository characteristics distinguished by topics. Newest version files of all plugin repository workflows are managed in cloudforet-io/actions.

Step 2-2. spaceone/actions/.github/worflows/deploy.yaml runs push_build_dev.yaml in each plugin repository

  1. push_build_dev.yaml proceeds versioning based on current date.

  2. push_build_dev.yaml upload the plugin image in Docker.

  3. push_build_dev.yaml sends notification through Slack.



To build and release the docker image of plugin repositories, plugins use dispatch_release.yaml.

  1. dispatch_release.yaml in each plugin repository is triggered manually.

  2. dispatch_release.yaml executes condition_check job to check version format and debug.

  3. dispatch_release.yaml updates master branch version file.

  4. dispatch_release.yaml executes git tagging.

  5. dispatch_release.yaml builds and pushes to Docker Hub with docker/build-push-action@v1

  6. dispatch_release.yaml sends notification through Slack.



For further details, you can check our GitHub cloudforet-io/actions.


6 - Tools CI

Detailed Explanation of Tools Repository CI

Tools CI process details



spacectl, spaceone-initializer, tester repositories are tools used for the spaceone project. There are some differences from other repositories' CI process.


spacectl repository workflow includes test code for each push with a version tag, which is similar to the CI process of backend core repositories.


spaceone-initializer repository does not include the workflow file triggered by ‘master branch push’, which most of repositories including spacectl and tester have.


Tools-category repositories use different repositories to upload.

  • spacectl : PyPi and Docker both
  • spaceone-initializer : Docker
  • tester : PyPi


To check the details, go to the .github/workflow directory in each directory.