elm-sync Configuration

Overview

elm-sync configuration is done in the Spring Boot application.yml file. The configuration associates one CDCM space with one or more ELM GCM servers, and lists the CDCM Configuration Areas in that space that should be synchronized with ELM GCM. Syncing configuration areas in other CDCM spaces requires a separate instance of elm-sync.

Configuration includes the following topics:

• The elm-sync server configuration

• scheduler for elm-sync

• spring application configuration including

  • security for oauth2 for CDCM access

  • SQLite database configuration for elm-sync

• OAuth1.0a configuration for access to ELM GCM

• The cdcm Configuration Area to ELM GCM server mappings

Description

elm-sync uses the configuration information in the application.yml file to configure the CDCM and ELM GCM server URLs, access control and API information, and the list of CDCM Configuration Areas to sync. elm-sync then uses the CDCM TRS provider to read the change log to replicate changes in CDCM in the corresponding project areas and resources in ELM GCM. elm-sync does not use the CDCM TRS base to determine what configuration areas to sync. Rather the CDCM Configuration Areas to sync are explicitly defined in the configuration.area.mapping elements.

elm-sync also does not currently use the trs:cutoffEvent, as it is always 0. This effectively means elm-sync assumes the TRS base is empty and all the TRS provider information is in the change log. This will be updated when CDCM implements change log pruning.

Configuration

application.yml

The application.yml defines various information elm-sync uses to determine what CDCM configuration areas should be synced to ELM GCM and how to access them through their authenticated APIs. Here’s an example application.yml file.

elm-sync:  # configuration for the elm-sync server
  instance:
    name: ELM-SYNC-1
  max-number-of-retries: 100
  retry-backoff: 100
  max-number-of-unique-title-retries: 100

server:
  port: 8080
  servlet:
    context-path: "/sandbox"

scheduler:
  cron: "0 * * * * *"

logging:
  level:
    root: info
    com:
      mid:
        smartfacts:
          cdcm:
            elmsync: debug
spring:
  codec:
    max-in-memory-size: 10MB
  security:
    oauth2:
      client:
        registration:
          cdcm-client:
            client-id: cdcm-client-id
            client-secret: secret
            scope: service-user-roles
            authorization-grant-type: client_credentials
        provider:
          cdcm-client:
            issuer-uri: https://keycloak/realms/cdcm
            user-name-attribute: preferred_username
      resourceserver:
        jwt:
          issuer-uri: https://keycloak/realms/cdcm
  datasource:
    url: "jdbc:sqlite:/opt/elmsync/data/elmsync.db"
    driver-class-name: org.sqlite.JDBC
    username: sa
    password: sa
  jpa:
    database-platform: org.hibernate.community.dialect.SQLiteDialect
    hibernate:
      ddl-auto: update
    show-sql: false

smartfacts:
  oauth10a:
    active: true
    outbound:
      details:
        - name: elm.com
          protected-url-roots: https://elm.com:9443/jts/**,https://elm.com:9443/rm/**,https://elm.com:9443/gc/**
          consumer-key: key
          consumer-secret: secret
          rootservices: https://elm.com:9443/jts/rootservices

    inbound:
      realm-name: sandbox-realm
      auto-approve-consumer-keys: true
      auto-approve-tokens: true
      details:
        - name: elm-inbound
          consumer-key: key
          consumer-secret: secret

configuration:
  area:
    mapping:
      - source: https://cdcmURL/spa/spaces/${spaceKey}/area/${areaId}
        target: https://elm.com:9443/gc/process/project-areas/${projectId}
      - source: https://cdcmURL/spa/spaces/${spaceKey}/area/${areaId}
        target: https://elm.com:9443/gc
#      - source: https://integration.dev.cdcm.smartfacts.com/spa/spaces/CDCM-IPKvRuZYUps1/area/65ddf34582b9156e1c1c5634
#        target: https://elmdemo.smartfacts.com:9443

elm-sync server configuration

This section defines the elm-server configuration including the server port and context-path for accessing the elm-sync controllers, and server instance information. Note: the elm-sync controllers are only used for development and unit testing, the only elm-sync REST services intended for customer use are for sync error reporting and recovery as described in Trouble Shooting .

elm-sync:  # configuration for the elm-sync server
  instance:
    name: ELM-SYNC-1
  max-number-of-retries: 6
  retry-backoff: 2
  max-number-of-unique-title-retries: 50

server:
  port: 8080
  servlet:
    context-path: "/sandbox"

elm-sync Spring Boot Application Configuration

The elm-sync Spring Boot application configuration specifies information about:

1. CDCM OAuth2.0 security information needed to access any CDCM spaces

2. An SQLite database for persisting information about the CDCM configuration areas and ELM GCM project areas that are being synchronized.

CDCM Authentication Configuration

Access to CDCM uses AOuth 2 which is configured for the Spring Boot application using the cdcm-client. Your CDCM administrator can provide your client-id and client-secret, and information on the provider.cdcm-client that does the authentication.

spring:
  codec:
    max-in-memory-size: 10MB
  security:
    oauth2:
      client:
        registration:
          cdcm-client:
            client-id: cdcm-client-id
            client-secret: secret
            scope: service-user-roles
            authorization-grant-type: client_credentials
        provider:
          cdcm-client:
            issuer-uri: https://keycloak/realms/cdcm
            user-name-attribute: preferred_username
      resourceserver:
        jwt:
          issuer-uri: https://keycloak/realms/cdcm

The oauth2.client.registration.cdcm-client configurations in the application.yml file are used for accessing CDCM.

For each client.registration, there is a corresponding client.provider that configures the authentication provider and method for that client.registration.

elm-sync Authentication Configuration

elm-sync provides a controller UI for accessing and repairing CDCM configuration area to GCM project area mappings as described in Trouble Shooting. This requires users to authenticate with elm-sync. See https://docs.spring.io/spring-security/reference/servlet/oauth2/resource-server/jwt.html for how this is done in a spring boot application. The spring.security.oauth2.resourceserver specifies how elm-sync is authenticated.

      resourceserver:
        jwt:
          issuer-uri: https://keycloak/realms/cdcm

Database Configuration

The database information includes:

• The Configuration Area id

• The ELM Server URL and corresponding project area URL

• The trs:order number of the last trs:change event processed by elm-sync. Changes after this event will be processed on the next elm-sync scan cycle.

spring:
  datasource:
    url: "jdbc:sqlite:/opt/elmsync/data/elmsync.db"
    driver-class-name: org.sqlite.JDBC
    username: sa
    password: sa
  jpa:
    database-platform: org.hibernate.community.dialect.SQLiteDialect
    hibernate:
      ddl-auto: update
    show-sql: false

ELM GCM OAuth1.0a Security Configuration

elm-sync uses OAuth1.0a to access ELM GCM through the REST APIs. This access is done through a functional user whose ID is associated with a consumer key and secret.

smartfacts:
  oauth10a:
    active: true
    outbound:
      details:
        - name: elm.com
          protected-url-roots: https://elm.com:9443/jts/**,https://elm.com:9443/rm/**,https://elm.com:9443/gc/**
          consumer-key: key
          consumer-secret: secret
          rootservices: https://elm.com:9443/jts/rootservices

    inbound:
      realm-name: sandbox-realm
      auto-approve-consumer-keys: true
      auto-approve-tokens: true
      details:
        - name: elm-inbound
          consumer-key: key
          consumer-secret: secret

The outbound.details provides the information elm-sync needs to be able to access ELM GCM services using OAuth1.0a for authentication.

• name: a name for the outbound.detals for a specific ELM GCM server configuration. There could be many of these, one for each ELM GCM server used in the configuration mapping

• protected-url-roots: lists the server URL roots that are protected by this outbound.details entry

• consumer-key: the consumer key associated with the functional id

• consumer-secret: the consumer secret associated with the consumer-key

• rootservices: the URL to the ELM server’s root services document that provides OAuth1.0a authentication details needed to authenticate a connection

The smartfacts.oauth10a.inbound is required, but not currently used. It is included for future use.

Creating the ELM GCM functional id

elm-sync access the ELM GCM REST APIs using OAuth1.0a and a functional id. The is the most reliable way to do server to server communication with ELM servers because they all use OAuth1.0a to configure consumer/friend relationships to allow the ELM servers to talk to each other. OAuth1.0a can be reliably depended on to be supported by ELM servers.

For elm-sync to access ELM GCM, you must register elm-sync with the ELM Jazz Team Server (JTS) as a consumer (inbound) connection, with consumer key and secret. Then you need to associate that consumer with a functional id that has JazzAdmins, JazzUsers and JazzProjectAdmins repository permissions. Here a brief summary of the steps.

1. Navigate to the Jazz Team Server Administration Home page

2. Manage or create users and create the functional id with JazzAdmins, JazzUsers and JazzProjectAdmins repository permissions

   

3. Click Manage Server and then Communication > Consumers (Inbound)

4. Use the Register Consumer form to create a Trusted Consumer Key with a Consumer Secret. This is the same information that will be used in the smartfacts.oauth10a.outbound.details entry in the application.yml file.

5. Edit the newly created Authorized Key and set the Functional User ID to the Id you create above:

   

If you create GCM project areas manually, you must add the functional id as an Administrator for the project area, and a member of the project are with the following roles: Administrator, Baseline Maker, Configuration Lead, Contributor, Tag Manager.

When creating the GCM project areas manually, carefully check the following:

1. For the CDCM configuration.area.mappings specified in the application.yml file, ensure the GCM project areas are created in the target GCM servers with exactly the same name (not ID) as the CDCM configuration area.

2. Make sure the functional id associated with the smartfacts.oauth10a.outbound.details.consumer-key is an Administrator and member of the GCM project areas with the Administrator, Baseline Maker, Configuration Lead, Contributor, Tag Manager process roles.

CDCM Configuration Area to ELM GCM Server Mappings

This section maps the CDCM configuration area to the ELM GCM server and optionally a specific project area.

configuration:
  area:
    mapping:
      - source: https://cdcmURL/spa/spaces/${spaceKey}/area/${areaId}
        target: https://elm.com:9443/gc/process/project-areas/${projectId}
      - source: https://cdcmURL/spa/spaces/${spaceKey}/area/${areaId}
        target: https://elm.com:9443/gc
#      - source: https://integration.dev.cdcm.smartfacts.com/spa/spaces/CDCM-IPKvRuZYUps1/area/65ddf34582b9156e1c1c5634
#        target: https://elmdemo.smartfacts.com:9443

The configuration.area.mapping.sourceis the URL of a CDCM configuration area. The configuration.area.mapping.target is either the base URL of an IBM GCM server, or the URL of a GCM project area.

If the target is a GCM base URL, then it is assumed that the elm-sync functional ID has JazzAdmins repository permissions which are required to be able to create and modify GCM project areas. If the target is a GCM project area, then elm-sync will reuse that existing project area, and JazzAdmins repository permissions are not required. In this case, elm-sync assumes ownership of the components in that GCM project area, and will rename and archive existing components that have the same name as the corresponding CDCM Unit before recreating the component.

The configuration area mappings can reference multiple CDCM servers and spaces, and the same CDCM configuration area can be mapped to multiple GCM servers, allowing the CDCM configuration information to be used in different GCM servers.

Configuration

elm-sync is configured using the Spring Boot project file application.yml. The location of this files specified in the docker-compose.yaml file. Spring Boot profiles can be used for different configurations such as application-local.ymlfor local development, and application-prod.yml for production.

Kubernetes

The application.yml contains security sensitive information such as consumer keys and secrets, and should be placed into a kubernetes secret, and mounted into the elm-sync container. The name of this secret is "elm-sync", the value of is the application.yml file.

To integrate the application.yml file into your elm-syn deployment, you need to create a secret called “elm-sync” in the namareaaespace of your elm-sync deployment.

There are two ways to do this:

1. Use kubectl

kubectl create secret generic elm-sync --from-file=application.yml=./resources/application.yml -n cdcm

2. If the secret has to be created manually or from a vault, use this template:

apiVersion: v1
data:
  application.yml: <base64 encoded content of the file application.yml>
kind: Secret
metadata:
  name: elm-sync
  namespace: cdcm
type: Opaque

Save the file as application.yml and apply it with:

kubectl apply -f application.yml -n <namespace>

Reference

The following tables define each of the properties in the elm-sync configuration that users will need to provide. See the example above for the property paths.

elm-sync Server Configuration: elm-sync, server

elm-sync Persistance: datasource

CDCM Security: spring.security.oauth2

Specifies the CDCM OAuth 2 security information.

ELM GCM Security Configuration: smartfacts.oauth10a

Provides the information needed for elm-sync to authenticate with ELM GCM and JTS using OAuth1.0a.

CDCM Configuration Area Mappings: configuration.area.mapping

This is a list of the CDCM configuration area IDs to ELM GCM server URI mappings.

Links

• RFC 6749: The OAuth 2.0 Authorization Framework

• OAuth 1.0a Specifications

• RFC 7617: The 'Basic' HTTP Authentication Scheme

• Gateway Proxy