Versions Compared

Version Old Version 66 New Version Current
Changes made by

Rene Beckert

Cornelius Wachinger

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Table of Contents
maxLevel7
minLevel1

Preparations

If not done so far please prepare your infrastructure as described in https://smartfacts.atlassian.net/wiki/spaces/SPD/pages/54624257/Installing+Smartfacts+for+Proof+of+Concepts#Prepare-your-infrastructure.

Setup the Kubernetes Environment

You need a Kubernetes cluster to deploy Smartfacts in. If you already have a Kubernetes infrastructure on AWS, GCP or Azure, you can use this.

...

If you don’t have a Kubernetes infrastructure you can set up a single node cluster at a vm. The following section will describe how to set zu such a cluster.

Install k3s

If your server only reaches the Internet via a proxy, you must set up your environment so that this proxy is used with k3s and helm. If this is the case, follow the instructions in this section:

...

Code Block
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.27.9+k3s1 sh -

Wait 30 seconds.

You can test it with the command sudo kubectl get node. It will provide an output similar to this one:

...

For more information see: https://rancher.com/docs/k3s/latest/en/quick-start

Install helm

Code Block
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && chmod 700 get_helm.sh && ./get_helm.sh

For more information see: https://helm.sh/docs/intro/install

Set Link to Cluster Configuration

Code Block
sudo mkdir -p /root/.kube
sudo ln -s /etc/rancher/k3s/k3s.yaml /root/.kube/config

Prepare Deployment

Add MID Repository

You will need a username and password to download the helm chart and containers from MID's repository. You will receive username and password from your MID contact or from cops@mid.de.

...

Code Block
sudo helm repo update

Provide Certificates and Key as Base64 String

Make sure that the complete certificate chain is contained in a file in correct order in x509 PEM format. The content of certificates in PEM formate looks like this:

...

  • server certificate at the top

  • intermediate cerificate(s), if any

  • root certificate at the bottom

Also see Section “Certificates” above in this article.

...

Expand
titleIf you received a .pfx file

Run the following command to extract everything from the .pfx file:

Code Block
cat <your
openssl pkcs12 -in certificate
chain
.
crt>
pfx
| base64
-out full.pem -nodes

Save certificate chain as a base64 string:

Code Block
cat <your certificate chain.crt> | base64 -w0 > chain.crt.base64

Save certificate key as base64 string:

Code Block
cat <your certificate key.key> | base64 -w0 > key.base64

Customize Values File

  • Change to your user's home directory on the server and create a new text file values.yaml with the following content:

    Code Block
    global:
  domain: "<your-domain>"
  instance: "smartfacts-poc"
  registry: "repo.mid.de"
  repologin: <login provided by MID>
  certsecrets:
    crtFullChainoauth2: "<FULL-CHAIN-CERT-BASE64>"
      keytokenClaims:
"<CERT-KEY-BASE64>"   ingress:     enabledemail: true"" # optionally specify the #claim Smartfactsfrom supportstoken Traefikor anduser nginxinfo aswhich Ingresscontains Controller.the email address of the user # The default is Traefik.(default: "[token]:email")
       # Ifid: you"" use# nginxoptionally asspecify ingressthe controllerclaim pleasefrom commenttoken inor thisuser line:info which contains the ID  #ingressClassTypeof the user (default: "nginx"[token]:sub")
  spacert:
    cspConfcrtFullChain: "<FULL-CHAIN-CERT-BASE64>"
    externalUrlskey: "<CERT-KEY-BASE64>"
 # spaceingress:
separated list of your toolenabled: urlstrue
(IBM Jazz, Jama, Codebeamer) plugin:# Smartfacts supports importPlugins:Traefik trueand #nginx "true"as forIngress firstController. installation,
"false" for minor updates will# speedThe updefault theis updateTraefik.
process a lot genoslc: # If enabled:you trueuse #nginx Setas toingress falsecontroller ifplease youcomment wantin tothis useline:
Smartfacts only (no OSLC connection to itger tools). In this case you cann skip the rest of this "genoslc" section.
  env:
    trsEnabled: false # Set to true if you use Smartfacts as data source for a link index
    configuration:
      # Give a list of up to 5 usernames of users which will have the administration right to change settings in Smartfacts.
      # At leas one administrator user must be stated here.
      administrators: 
        - acm # replace this value with the username of the person who should configure the oauth10a information for the OSLC Connector for Smartfacts#ingressClassType: "nginx" 
spa:
  cspConf:
    externalUrls: "" # space separated list of your tool urls (IBM Jazz, Jama, Codebeamer)
plugin:
  importPlugins: true # "true" for first installation, "false" for minor updates will speed up the update process a lot
genoslc:
  enabled: true # Set to false if you want to use Smartfacts only (no OSLC connection to other tools). In this case you can skip the rest of this "genoslc" section.
  env:
    trsEnabled: false # Set to true if you use Smartfacts as data source for a link index
    configuration:
      # Give a list of up to 5 usernames of users which will have the administration right to change settings in Smartfacts.
      # keyAt usedleas toone administrator encryptuser themust oauth10abe configurationstated datahere.
      oauth10aEncryptionKeyadministrators: ""
# define the key which is used to encrypt- theacm oauth10a# informationreplace inthis thevalue databasewith mailservice:the username of enabled:the falseperson camp:who should configure enabled:the falseoauth10a kafka:information for  enabled: false
    Replace the value of the domain property with your domain name.
    the OSLC Connector for Smartfacts
      # key used to encrypt the oauth10a configuration data
      oauth10aEncryptionKey: "" # define the key which is used to encrypt the oauth10a information in the database
mailservice:
  enabled: false
camp:
  enabled: false
kafka:
  enabled: false

  • Replace the value of the repologin domain property with the value provided from MID.Replace the your domain name.

  • If your cluster has access to the internet, then you can pull the images directly from the registry provided by MID. In this case replace the value of the repologin property with the value provided from MID.

  • Replace the placeholder of the crtFullChain property with the content of the file chain.crt.base64.

  • Replace the placeholder of the key property with the content of the key.base64 file.

  • Replace the placeholder for the externalUrls property with the URL of your Jama or Codebeamer instance. If you connect multiple instance delement instances, separate the URLs with a space.

  • Save the values.yaml file.

Install Smartfacts

Execute the Installation

Enter the following command to execute the Installation:

...

Option: Use a private container registry

If your Kubernetes cluster has no access to the internet, you can pull the images from the MID registry, re-tag them and push them to your private registry. State the name of your private registry in the property

Code Block
global:
  [...]
  registry: <Name of your private registry>

in your values file.

Install Smartfacts

Execute the Installation

Enter the following command to execute the Installation:

Code Block
sudo helm upgrade --install --timeout 20m0s smartfacts mid-smartfacts/smartfacts -f values.yaml -n smartfacts --create-namespace

Watch Deployment (in a new Session)

Open a second session on the server and enter the following command:

...

As soon as all deployments are available, the installation is ready.

Add Web Origin to Keycloak

Note

For Smartfacts to function, it is necessary to corret correct a value in the Keycloak configuration.

...

Call this URL in a browser. Klick Click on “Administration Console” and log in with Username “keycloak-admin” and the password returned in the step above.

...

Scroll to the bottom of the page and klick click on the “save” button

A note on Smartfacts PoC Test Users

For the Smartfacts PoC you can use the generated test users “poc1”…”poc50”. All poc - users have the same password “poc”. For managing the test users in the CAMP use the account administrator user “acm” which has the password “acm”.

Create a Smartfacts Model Warehouse

  1. Open the Smartfacts URL in your browser: https://smartfacts.<yoursmartfactsdomain> and log-in with the account manager credentials (user “acm”, password “acm”).

  2. You will now be asked to create a Model Warehouse.

    1. Click on CREATE A NEW MODEL WAREHOUSE.

    2. In the create dialog, first select the previously created account.

    3. Provide a name for the Model Warehouse.

    4. Optionally, activate the check box Create Demo Models if you want some demo content to be added to the Model Warehouse.

Model Warehouse creation is the final step in the Smartfacts platform set-up process.

Install the Plug-Ins

It depends on which tools you want to use in connection with Smartfacts. Smartfacts provides a plug-in plugin for every tool which is supported. You can download the plug-ins from the main menu via the command Get Plug-ins.

Troubleshooting

For general Kubernetes Troubleshooting you can consult the https://kubernetes.io/docs/reference/kubectl/cheatsheet/.

Detect running containers

Code Block
sudo kubectl -n smartfacts get pods

Show the logs of a pod

With the following command you can get the log files of the Smartfacts platform (pod “sfit-platform”). If you deployed with a namespace named other than “smartfacts”, adjust the value of the second variable “ns”.

...

value for pod variable

POD

sfit-platform

The Smartfacts platform

sfit-spa

The Smartfacts SPA

mongodb

The mongodb database

genoslc

The OSLC Connector for Smartfacts

keycloak

The keycloak pod (identity provider for PoC installations with dummy users)

Show certtool logs

Show certtool output from Job

...

Code Block
sudo kubectl get secret smartfacts-truststore -o 'go-template={{index .data "certtool.log"}}' -n smartfactsmartfacts | base64 -d

Show Keycloak password

If it is necessary to login into the admin UI of Keycloak, the password for the admin user “keycloak-admin” can be optained by the following command.

Code Block
sudo kubectl get secret smartfacts-keycloak-admin-secret -o 'go-template={{index .data "KEYCLOAK_PASSWORD"}}' -n smartfacts | base64 -d; echo ""

...

Reset lost password for keycloak-admin user

In rare cases, it might happen that the password of the keycloak-admin user is different from the password in the secret “smartfacts-keycloak-admin-secret” and is not known anymore. This section describes the steps necessary to reset this password.

To reset the password for the keycloak-admin user:

  1. Get the password from the secret like described in the section “Show Keycloak password” above. Save the password in a password manager.

  2. Edit the secret “smartfacts-keycloak-admin-secret”, this command will open the secret in a vi editor:

    Code Block
    sudo kubectl edit secret smartfacts-keycloak-admin-secret -n smartfacts

    1. replace the value of the field “KEYCLOAK_USER” with the base64 value for “tmp-admin”: dG1wLWFkbWlu

    2. Save and quit

  3. Restart the keycloak pod:

    Code Block
    pod=keycloak ns=smartfacts bash -c 'kubectl delete pod $(kubectl get pods -n $ns | grep $pod | cut -d " " -f1) -n $ns'

  4. Sign in into the keycloak UI with the user “tmp-admin” and the password saved from the secret

  5. In the Keycloak UI, switch to the realm “Master” and then to the section “users”

    1. Klick on “View all users”

    2. Klick on the ID of the user “keycloak-admin”

      image-20240723-165248.pngImage Added

    3. Go to section “Credentials” and insert the correct password in the fields in section “Reset Password”

      1. Switch “Temprory” to “OFF”

        image-20240723-165616.pngImage Added

      2. Klick on “Reset Password” and confirm the Reset Password dialog

    4. Log off of Keycloak

  6. Edit the secret “smartfacts-keycloak-admin-secret” again

    Code Block
    sudo kubectl edit secret smartfacts-keycloak-admin-secret -n smartfacts

    1. replace the value of the field “KEYCLOAK_USER” with the base64 value for “keycloak-admin”: a2V5Y2xvYWstYWRtaW4=

    2. Save and quit

  7. Restart the keycloak pod

    Code Block
    pod=keycloak ns=smartfacts bash -c 'kubectl delete pod $(kubectl get pods -n $ns | grep $pod | cut -d " " -f1) -n $ns'

  8. Sign in into keycloak with keycloak-admin and the password saved from the secret

    1. In the Keycloak UI switch to the realm “Master” and then to section “users”

      1. Klick on “View all users”

      2. Delete the user “tmp-admin”

        image-20240723-165203.pngImage Added

  9. log out of Keycloak

How to Move k3s data to another partition or disk
Insert excerpt
Excerpt: How to move k3s data to another partition or disk
Excerpt: How to move k3s data to another partition or disk
namemove k3s to other disk
nopaneltrue

Add Certificates to the Java Truststore

If Java services need to make requests to other infrastructure services (e.g. the identity provider of you company), they need to trust the certificates of these services. To trust these certificates it is necessary to have their intermediate and root certificates in the truststore of the Smartfacts installation.

This chapter describes how to add additional certificates into this truststore. The steps that require a Java installation will be done inside the sfit-platform container using its Java installation.

Note

In this guide, we assume that the sfit-platform POD runs stable. If it restarts or crashes, use a Linux server in your company with Java installed to create the truststore.

  1. In your values.yaml file, disable the certtool. Normally the certtool runs at the beginning of the deployment process and re-creates the truststore. This would overwrite our manual changes.
    To disable the certtool insert the following snippet into your values.yaml file (or set the property to false if it already exists):

    Code Block
    certtool:
  enabled: false

  2. Copy the intermediate or root certificates that should be added to the truststore to your PC with access to the cluster.

  3. Get the name of the sfit-platform pod with this command.

    Code Block
    sudo kubectl get pods -n smartfacts | grep sfit-platform | cut -d " " -f1

  4. Copy the certificates into the directory /tmp inside the container

    Code Block
    sudo kubectl cp <certificate_file> <sfit-platform_pod_name>:/tmp -n smartfacts

    replace <certificate_file> with the path to your certificate file on your PC and <sfit-platform_pod_name> with the result from the step 2.
    You can copy only one file with cp. If you need to copy multiple files, run it multiple times.

  5. Open a shell in the sfit-platform pod.

    Code Block
    sudo kubectl exec -it <sfit-platform_pod_name> -n smartfacts -- sh

  6. Copy the truststore to the directory /tmp and change to the tmp directory

    Code Block
    cp /opt/security/truststore.jks /tmp
cd /tmp

Info

If the file /opt/security/truststore.jks doesn`t exist, you can copy it from the path /etc/ssl/certs/java/cacerts with

Code Block
cp /etc/ssl/certs/java/cacerts /tmp/truststore.jks
Info

If you are on a different Linux machine with Java installed and the cacerts you can copy the cacerts file with this command:

Code Block
cp $(readlink -f /usr/bin/java | sed "s:bin/java::")/lib/security/cacerts /tmp/truststore.jks

  1. Add your certificate to the truststore

    Code Block
    keytool -import -alias <your_certificate_alias> -file <certificate_file> -storetype JKS -keystore truststore.jks

    Replace the placeholder <your_certificate_alias> with an alias of the certificate in the truststore.
    Replace the placeholder <certificate_file> with the filename of your certificate file.
    Enter the password of the truststore (most likely it is “changeit”; If it’s not, look it up in the smartfacts-truststore-password secret)
    Answer the question “Trust this certificate?” with “yes”

    If you need to add more than one certificate repeat this step for all your certificates.

  2. Leave the container with exit.

  3. Copy the extended truststore back to your PC.

    Code Block
    sudo kubectl cp <sfit-platform_pod_name>:/tmp/truststore.jks ./extended_truststore.jks -n smartfacts

  4. Remove the secret “smartfacts-truststore” (if it is already present)

    Code Block
    sudo kubectl delete secret smartfacts-truststore -n smartfacts

  5. Re-create the secret “smartfacts-truststore”

    Code Block
    sudo kubectl create secret generic smartfacts-truststore --from-file=truststore.jks=./extended_truststore.jks -n smartfacts

  6. If there is not already a secret “smartfacts-truststore-password” present, create it with

    Code Block
    sudo kubectl create secret generic smartfacts-truststore-password --from-literal=password=changeit -n smartfacts

    Adopt the password if the truststore you copied has a different one.

  7. Restart the sfit-platform POD, so that it uses the new truststore (As soon as we delete the pod, the deployment will automatically create a new one with our changes loaded)

    Code Block
    sudo kubectl delete pod <sfit-platform_pod_name> -n smartfacts

Keep the file “extended_truststore.jks”, so that you can use it to re-build the smartfacts-truststore secret anytime.

Postgres Password – How to Fix the "'global.postgresql.auth.postgresPassword' must not be empty" Error

During Smartfacts installation, some users may face an unexpected issue with Postgres, which is part of the Keycloak service. This issue may occur in different scenarios, such as syntax, certificate or repologin errors in a values.yaml file, network issues etc.

...