Configure MAIA Agent for git scanning

Prerequisites
  1. Install MAIA Agent on the host where it will run as a git scanner. The git scanner scans one or more git repositories, and posts commits to the MAIA WebApp. The repositories will be cloned as a part of the scan process. Thus, they must be accessible from the host where the scanner is running, and a user (scan user) with "read access" to the git repositories is required.

  2. Create another user in MAIA WebApp.

    • The user must be permitted to access MAIA WebApp with http requests from the host.
    • The user must also be permitted to access Kafka from the host.
  3. Add user to an existing team in MAIA or create a team first.

  4. Log in to the MAIA WebApp as user and go to the user show page, create an API-token.

  5. Set permissions for the user (or the team it belongs to).

    The user must have Access all records and Create commits permissions set.

Workflow with examples
  1. Decide on a scan user that is responsible for the git scanning according to the above requirements. Our example user is 'maia' belonging to team 'GitScan'

  2. Decide on where the scanner shall execute (host and directory). In our example we use the MAIA WebApp host 'install-test1' and directory '/opt/scan'.

    The git scanner can run anywhere as long as the requirements are met

  3. Store the scan user access token locally in the home account on the scan host

    maia@install-test1:~$ echo <MY ACCESS TOKEN FOR 'scan user-team'> ~/.maia-api-token
  4. Create a top scan directory for the execution where the scan user has write and execution access.

    maia@install-test1:~$ mkdir -p /opt/scan
  5. As the scan user, create an environment.yml file in the top scan directory.

    maia@install-test1:~$ touch /opt/scan/environment.yml
  6. The scan user must have write access to LOG_ROOT. Create the directory if necessary.

    maia@install-test1:~$ mkdir -p /opt/scan/logs
  7. Edit environment.yml and add the parameters required for git scan.

    Example

    LOG_ROOT: /opt/scan/logs
    MAIA_CORE_HOST: install-test1.example.com
    MAIA_CORE_HTTP_PORT: 8888
    MAIA_CORE_APP_CONTEXT: /maia
    MAIA_CORE_API_TOKEN_FILE: ${home}/.maia-api-token
    MAIA_CORE_USE_SSL: false
    GIT_SCAN_TRACKED_FILE: commits_tracked
    REPOSITORIES:
     (.*): ssh://maia@grit.example.com:29418/$1
    KAFKA_BROKERS:
    - install-test1.example.com:9092
    KAFKA_TOPIC: MAIA
  8. Export the variables in the environment file to BLT_ENV.

    export BLT_ENV=<path_to_maia_agent>/environment.yml
    
    Example:
    export BLT_ENV=/opt/scan/environment.yml
  9. Scan repositories defined in the file environment.yml.

    To scan repositories:

    • Run the command maia git_scan for a single repository.

      bundle exec maia git_scan --repo-name <repository_name> -v
      
      Example:
      bundle exec maia git_scan --repo-name myrepo -v

    OR

    • Run the command maia git_scan for multiple repositories.

      As the scan user, create a repository definition file.

      touch <path_to_maia_agent>/repodefs
      
      Example
      touch /opt/scan/repodefs

      Edit the repository definition file and add the repositories, one per line. Example of file content:

      unicorn
      redis
      init_common
      yarn
      chromedriver

      Run the command maia git_scan

      bundle exec maia git_scan ---repo-file <path_to_maia_agent>/repodefs -v
      
      Example
      bundle exec maia git_scan ---repo-file /opt/scan/repodefs -v
  10. Verify that the commits in the repositories are scanned into MAIA application.

    • Log in to MAIA application as the root admin user admin.
    • On the dashboard, verify that commits are visible in the Latest Commits widget.

Below a description of common scanning error messages

plugin:content-inject