This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Top Level Operator

Top Level Operator.

1 - Customer Notifications

Status Page

https://gitlab.cee.redhat.com/service/app-interface/-/blob/master/docs/app-sre/statuspage.md

https://service.pages.redhat.com/dev-guidelines/docs/appsre/advanced/statuspage/

Service Logs

https://gitlab.cee.redhat.com/service/managed-tenants-sops/-/blob/main/MT-SRE/sops/mt-sre-customer-notification.md

Internal Email

There are multiple ways a user or group can get notified of service events (e.g. planned maintenance, outages). There are two fields in the addon metadata file (see Add-On metadata file schema documentation for more information) where email addresses can be provided:

  • addonOwner: REQUIRED Point of contact for communications from Service Delivery to addon owners. Where possible, this should be a development team mailing list (rather than an individual developer).
  • addonNotifications: This is a list of additional email addresses of employees who would like to receive notifications about a service.

There is also a mailing list that receives notifications for all services managed by Service Delivery. Subscribe to the sd-notifications mailing list here.

2 - Dependencies

This document describes the supported implementation for Addon dependencies, as signed-off by the Managed Tenants SRE Team.

Dependencies Specification

  • Addons must specify dependencies using the OLM dependencies feature, documented here
  • The dependencies must have the version pin-pointed. Ranges are not allowed.
  • The dependencies must come from a Trusted Catalog. See the Trusted Catalogs section for details.

Trusted Catalogs

The Addon and its dependencies must come from Trusted Catalogs. Trusted Catalogs are those with content published by the Managed Services Pipelines, implemented by CPaaS, or by the Managed Tenants SRE Team.

Trusted Catalogs List

  • Addon catalog: the catalog created by the Managed Tenants SRE Team, for the purpose of releasing the Addon. Dependency bundles can be shipped in the same catalog of the Addon. The Addon catalog is considered “trusted” for the dependencies it carries.
  • Red Hat Operators catalog: the catalog content goes through the Managed Services Pipelines, same process to build some Addons themselves, just with a different release process. This catalog is considered “trusted” and can be used for dependencies.

Including a Catalog in the Trusted List

  • Make sure that the catalog is available on OSD and its content is released through the Managed Services Pipelines, implemented by CPaaS.
  • Create a Jira ticket in the MT-SRE Team backlog, requesting the assessment of the OSD catalog you want to consider as “trusted”.

Issues

There’s a feature request to the OLM Team to allow specifying the CatalogSource used for the dependencies:

3 - Environments

Mandatory environments

Add-ons are normally deployed to two environments:

  • ocm stage: development/testing - All add-ons must deploy to this environment before being released to production.
  • ocm production: once the deployment in stage has been reviewed, accepted, and approved it can be promoted to production via /lgtm by your SRE team.

We recommend the ocm stage and ocm production add-on metadata be as similar as possible.

SLOs

ocm stage have no SLO and operates with best effort support from Add-on SRE, SREP, and App-SRE osd stage cluster have no SLO and operates with best effort support from Add-on SRE, SREP, and App-SRE ocm production environments are subject to App-SRE SLOs. osd production cluster environments are subject OSD SLOs.

Additional Environments (via duplicate add-ons)

Some add-on providers have had use cases which require additional add-on envs. While we only have ocm stage and ocm prod, managed-tenants may be leveraged to deploy to an additional add-on (like edge or internal). Today we don’t recommend this practice due to the need to clone all add-on metadata which increases the risk for incorrect metadata going to production/customer clusters.

If you need to do the above, please reach out to your assigned SRE team for guidance first.

4 - Plug and Play Addon

Package Operator

Package Operator is a Kubernetes Operator for packaging and managing a collection of arbitrary Kubernetes objects.

Each addon with a packageOperator defined in its spec will have a corresponding ClusterObjectTemplate. The ClusterObjectTemplate is an API defined in Package Operator, enabling users to create an object by templating a manifest and injecting values retrieved from other arbitrary source objects. However, regular users typically do not need to interact with the ClusterObjectTemplate. Instead, they can interact with the generated ClusterPackage manifest.

Example of a ClusterPackage manifest:

apiVersion: package-operator.run/v1alpha1
kind: ClusterPackage
metadata:
  name: <addon_name>
spec:
  image: <addon.spec.packageOperator>
   config:
    addonsv1:
      clusterID: a440b136-b2d6-406b-a884-fca2d62cd170
      deadMansSnitchUrl: https://example.com/test-snitch-url
      ocmClusterID: abc123
      ocmClusterName: asdf
      pagerDutyKey: 1234567890ABCDEF
      parameters:
        foo1: bar
        foo2: baz
      targetNamespace: pko-test-ns-00-req-apy-dsy-pdy

  • The deadMansSnitchUrl and pagerDutyKey are obtained from the ConfigMaps using their default names and locations. IMPORTANT: To successfully inject the deadMansSnitchUrl and pagerDutyKey values into the ClusterPackage manifest, you must keep the default naming scheme and location of the corresponding ConfigMaps. See the addons deadMansSnitch and addons pagerDuty documentation for more information.

  • Additionally, all the values present in .spec.config.addonsv1 can be injected into the objects within your packageImage. See the package operator documentation for more information.

Tenants Onboarding Steps

Although you can generate the packageImage yourself using the package operator documentation, we recommend you use the Managed Tenants Bundles (MTB) facilities.

The following steps are an example of generating the packageImage for the reference-addon package using the MTB flow:

  1. In the MTB repository, create a package directory and add the manifests.yaml inside the package directory. See the following merge request for an example.

  2. The MTB CI creates the packageImage and the Operator Lifecycle Manager (OLM) Index Image as part of the team’s addon folder.

  3. The MTB CI creates a merge request to the managed-tenants repository and adds a new AddonImageSet with the PackageImage and OLM Index images.