Configure MAIA Agent for git scanning

Prerequisites

1. Install MAIA Agent on the host where it will run as a git scanner. The git scanner scans one or more git repositories, and posts commits to the MAIA WebApp. The repositories will be cloned as a part of the scan process. Thus, they must be accessible from the host where the scanner is running, and a user (scan user) with "read access" to the git repositories is required.

2. Create another user in MAIA WebApp.

   • The user must be permitted to access MAIA WebApp with http requests from the host.
   • The user must also be permitted to access Kafka from the host.

3. Add user to an existing team in MAIA or create a team first.

4. Log in to the MAIA WebApp as user and go to the user show page, create an API-token.

5. Set permissions for the user (or the team it belongs to).

   The user must have Access all records and Create commits permissions set.

Workflow with examples

1. Decide on a scan user that is responsible for the git scanning according to the above requirements. Our example user is 'maia' belonging to team 'GitScan'

2. Decide on where the scanner shall execute (host and directory). In our example we use the MAIA WebApp host 'install-test1' and directory '/opt/scan'.

   The git scanner can run anywhere as long as the requirements are met

3. Store the scan user access token locally in the home account on the scan host

   maia@install-test1:~$ echo <MY ACCESS TOKEN FOR 'scan user-team'> ~/.maia-api-token

4. Create a top scan directory for the execution where the scan user has write and execution access.

   maia@install-test1:~$ mkdir -p /opt/scan

5. As the scan user, create an environment.yml file in the top scan directory.

   maia@install-test1:~$ touch /opt/scan/environment.yml

6. The scan user must have write access to LOG_ROOT. Create the directory if necessary.

   maia@install-test1:~$ mkdir -p /opt/scan/logs

7. Edit environment.yml and add the parameters required for git scan.

   Example

   LOG_ROOT: /opt/scan/logs
   MAIA_CORE_HOST: install-test1.example.com
   MAIA_CORE_HTTP_PORT: 8888
   MAIA_CORE_APP_CONTEXT: /maia
   MAIA_CORE_API_TOKEN_FILE: ${home}/.maia-api-token
   MAIA_CORE_USE_SSL: false
   GIT_SCAN_TRACKED_FILE: commits_tracked
   REPOSITORIES:
    (.*): ssh://maia@grit.example.com:29418/$1
   KAFKA_BROKERS:
   - install-test1.example.com:9092
   KAFKA_TOPIC: MAIA

8. Export the variables in the environment file to BLT_ENV.

   export BLT_ENV=<path_to_maia_agent>/environment.yml
   
   Example:
   export BLT_ENV=/opt/scan/environment.yml

9. Scan repositories defined in the file environment.yml.

   To scan repositories:

   • Run the command maia git_scan for a single repository.

     bundle exec maia git_scan --repo-name <repository_name> -v
     
     Example:
     bundle exec maia git_scan --repo-name myrepo -v

   OR

   • Run the command maia git_scan for multiple repositories.

     As the scan user, create a repository definition file.

     touch <path_to_maia_agent>/repodefs
     
     Example
     touch /opt/scan/repodefs

     Edit the repository definition file and add the repositories, one per line. Example of file content:

     unicorn
     redis
     init_common
     yarn
     chromedriver

     Run the command maia git_scan

     bundle exec maia git_scan ---repo-file <path_to_maia_agent>/repodefs -v
     
     Example
     bundle exec maia git_scan ---repo-file /opt/scan/repodefs -v

10. Verify that the commits in the repositories are scanned into MAIA application.

    • Log in to MAIA application as the root admin user admin.
    • On the dashboard, verify that commits are visible in the Latest Commits widget.

Below a description of common scanning error messages

plugin:content-inject