feat(traits): gitops trait for Pipe

* Closes: #6421

Signed-off-by: Michal Vavřík <michal.vavrik@aol.com>

Author: michalvavrik

docs/modules/ROOT/pages/running/gitops.adoc

@@ -1,7 +1,7 @@
 [[gitops]]
 = Camel GitOps
 
-Once your build is complete, you can configure the operator to run an opinionated GitOps strategy. Camel K has a built-in feature which allow the operator to push a branch on a given Git repository with the latest Integration candidate release built. In order to set the context, this would be the scenario:
+Once your build is complete, you can configure the operator to run an opinionated GitOps strategy. Camel K has a built-in feature which allow the operator to push a branch on a given Git repository with the latest Integration or Pipe candidate release built. In order to set the context, this would be the scenario:
 
 1. The dev operator builds the application from Git source
 2. The dev operator push the container image
@@ -43,7 +43,31 @@ spec:
 
 NOTE: There are more options to configure on the `gitops` trait. Feel free to have a look and learn on the trait documentation page directly.
 
-As soon as the build of the Integration is completed, the operator will prepare the commit with the overlays. The structure would be like the following directory tree:
+The same trait can be used on a Pipe:
+
+```yaml
+apiVersion: camel.apache.org/v1
+kind: Pipe
+metadata:
+  name: timer-to-log
+spec:
+  source:
+    uri: timer:foo
+  sink:
+    uri: log:bar
+  traits:
+    gitops:
+      enabled: true
+      url: https://github.com/my-org/my-camel-apps.git
+      secret: my-gh-token
+      overlays:
+        - staging
+        - production
+```
+
+NOTE: When used with a Pipe, the `url` and `secret` trait parameters are required.
+
+As soon as the build is completed, the operator will prepare the commit with the overlays. For an Integration, the structure would be like the following directory tree:
 
 ```bash
 /integrations/
@@ -70,7 +94,30 @@ As soon as the build of the Integration is completed, the operator will prepare
 
 The above structure could be used directly with `kubectl` (eg, `kubectl apply -k /tmp/integrations/sample/overlays/production`) or any CICD capable of running a similar deployment strategy.
 
-The important thing to notice is that the **base** Integration is adding the container image that we've just built and any other trait which is required for the application to run correctly (and without the need to be rebuilt) on another environment:
+For a Pipe, the structure is similar but uses `pipe.yaml` and `patch-pipe.yaml` instead:
+
+```bash
+/pipes/
+├── all
+│   └── overlays
+│       ├── production
+│       │   └── kustomization.yaml
+│       └── staging
+│           └── kustomization.yaml
+└── timer-to-log
+    ├── base
+    │   ├── pipe.yaml
+    │   └── kustomization.yaml
+    └── overlays
+        ├── production
+        │   ├── kustomization.yaml
+        │   └── patch-pipe.yaml
+        └── staging
+            ├── kustomization.yaml
+            └── patch-pipe.yaml
+```
+
+The important thing to notice is that the **base** resource (Integration or Pipe) is adding the container image that we've just built and any other trait which is required for the application to run correctly (and without the need to be rebuilt) on another environment:
 
 ```yaml
 apiVersion: camel.apache.org/v1
@@ -184,7 +231,9 @@ NOTE: this is the approach suggested in https://developers.redhat.com/e-books/pa
 
 === Push to the same repository you've used to build
 
-If you're building application from Git and you want to push the changes back to the same, then, you don't need to configure the Git repository for `gitops` trait. If nothing is specified, the trait will get the configuration from `.spec.git` Integration. This approach may be good when you want to have a single repository containing all aspects of an application. In this case we suggest to use a directory named `ci` or `cicd` as a convention to store your GitOps configuration.
+If you're building application from Git and you want to push the changes back to the same repository, then, you don't need to configure the Git repository for `gitops` trait. If nothing is specified, the trait will get the configuration from `.spec.git` Integration. This approach may be good when you want to have a single repository containing all aspects of an application. In this case we suggest to use a directory named `ci` or `cicd` as a convention to store your GitOps configuration.
+
+NOTE: This does not apply to Pipes, which must always specify the `url` on the `gitops` trait.
 
 === Chain of GitOps environments
 

docs/modules/ROOT/partials/apis/camel-k-crds.adoc

@@ -7392,9 +7392,9 @@ The listeners in the format "port;protocol" (default, "8080;HTTP").
 
 * <<#_camel_apache_org_v1_Traits, Traits>>
 
-The GitOps Trait is used to configure the repository where you want to push a GitOps Kustomize overlay configuration of the Integration built.
+The GitOps Trait is used to configure the repository where you want to push a GitOps Kustomize overlay configuration of the Integration or Pipe built.
 If the trait is enabled but no pull configuration is provided, then, the operator will use the values stored in Integration `.spec.git` field used
-to pull the project.
+to pull the project. When used with a Pipe, the `url` and `secret` parameters are required as Pipes do not have a `.spec.git` fallback.
 
 
 [cols="2,2a",options="header"]

docs/modules/traits/pages/gitops.adoc

@@ -3,9 +3,9 @@
 // Start of autogenerated code - DO NOT EDIT! (badges)
 // End of autogenerated code - DO NOT EDIT! (badges)
 // Start of autogenerated code - DO NOT EDIT! (description)
-The GitOps Trait is used to configure the repository where you want to push a GitOps Kustomize overlay configuration of the Integration built.
+The GitOps Trait is used to configure the repository where you want to push a GitOps Kustomize overlay configuration of the Integration or Pipe built.
 If the trait is enabled but no pull configuration is provided, then, the operator will use the values stored in Integration `.spec.git` field used
-to pull the project.
+to pull the project. When used with a Pipe, the `url` and `secret` parameters are required as Pipes do not have a `.spec.git` fallback.
 
 
 This trait is available in the following profiles: **Kubernetes, Knative, OpenShift**.

pkg/apis/camel/v1/trait/gitops.go

@@ -17,9 +17,9 @@ limitations under the License.
 
 package trait
 
-// The GitOps Trait is used to configure the repository where you want to push a GitOps Kustomize overlay configuration of the Integration built.
+// The GitOps Trait is used to configure the repository where you want to push a GitOps Kustomize overlay configuration of the Integration or Pipe built.
 // If the trait is enabled but no pull configuration is provided, then, the operator will use the values stored in Integration `.spec.git` field used
-// to pull the project.
+// to pull the project. When used with a Pipe, the `url` and `secret` parameters are required as Pipes do not have a `.spec.git` fallback.
 //
 // +camel-k:trait=gitops.
 //

pkg/trait/gitops.go

@@ -20,8 +20,10 @@ package trait
 import (
 	"context"
 	"errors"
+	"fmt"
 	"os"
 	"path/filepath"
+	"strings"
 	"time"
 
 	v1 "github.com/apache/camel-k/v2/pkg/apis/camel/v1"
@@ -32,6 +34,7 @@ import (
 	"github.com/go-git/go-git/v5/plumbing"
 	corev1 "k8s.io/api/core/v1"
 	"k8s.io/utils/ptr"
+	ctrl "sigs.k8s.io/controller-runtime/pkg/client"
 
 	git "github.com/go-git/go-git/v5"
 	"github.com/go-git/go-git/v5/plumbing/object"
@@ -104,6 +107,15 @@ func (t *gitOpsTrait) pushGitOpsRepo(ctx context.Context, it *v1.Integration, to
 // to a new branch.
 func (t *gitOpsTrait) pushGitOpsItInGitRepo(ctx context.Context, it *v1.Integration, dir, token string) error {
 	gitConf := t.gitConf(it)
+
+	pipe, err := t.findOwnerPipe(ctx, it)
+	if err != nil {
+		return err
+	} else if pipe != nil && gitConf.URL == "" {
+		// Pipes have no .spec.git fallback, so the URL must be provided via the trait
+		return errors.New("gitops trait requires a git URL when used with a Pipe")
+	}
+
 	// Clone repo
 	repo, err := util.CloneGitProject(gitConf, dir, token)
 	if err != nil {
@@ -146,11 +158,21 @@ func (t *gitOpsTrait) pushGitOpsItInGitRepo(ctx context.Context, it *v1.Integrat
 		return err
 	}
 
-	for _, overlay := range t.getOverlays() {
-		destIntegration := util.EditIntegration(it, kit, overlay, "")
-		err = util.AppendKustomizeIntegration(destIntegration, ciCdDir, t.getOverwriteOverlay())
-		if err != nil {
-			return err
+	if pipe != nil {
+		for _, overlay := range t.getOverlays() {
+			destPipe := util.EditPipe(pipe, it, kit, overlay, "")
+			err = util.AppendKustomizePipe(destPipe, ciCdDir, t.getOverwriteOverlay())
+			if err != nil {
+				return err
+			}
+		}
+	} else {
+		for _, overlay := range t.getOverlays() {
+			destIntegration := util.EditIntegration(it, kit, overlay, "")
+			err = util.AppendKustomizeIntegration(destIntegration, ciCdDir, t.getOverwriteOverlay())
+			if err != nil {
+				return err
+			}
 		}
 	}
 
@@ -191,16 +213,37 @@ func (t *gitOpsTrait) pushGitOpsItInGitRepo(ctx context.Context, it *v1.Integrat
 	}
 
 	// Publish a condition to notify the change was pushed to the branch
+	resourceKind := v1.IntegrationKind
+	if pipe != nil {
+		resourceKind = v1.PipeKind
+	}
 	it.Status.SetCondition(
 		v1.IntegrationConditionType("GitPushed"),
 		corev1.ConditionTrue,
 		"PushedToGit",
-		"Integration changes pushed to branch "+branchName,
+		fmt.Sprintf("%s changes pushed to branch %s", resourceKind, branchName),
 	)
 
 	return nil
 }
 
+// findOwnerPipe checks if the Integration was created by a Pipe and returns it.
+func (t *gitOpsTrait) findOwnerPipe(ctx context.Context, it *v1.Integration) (*v1.Pipe, error) {
+	for _, o := range it.OwnerReferences {
+		if o.Kind == v1.PipeKind && strings.HasPrefix(o.APIVersion, v1.SchemeGroupVersion.Group) {
+			pipe := &v1.Pipe{}
+			key := ctrl.ObjectKey{Namespace: it.Namespace, Name: o.Name}
+			if err := t.Client.Get(ctx, key, pipe); err != nil {
+				return nil, fmt.Errorf("could not load Pipe %q: %w", o.Name, err)
+			}
+
+			return pipe, nil
+		}
+	}
+
+	return nil, nil
+}
+
 // gitConf returns the git repo configuration where to pull the project from. If no value is provided, then, it takes
 // the value coming from Integration git project (if specified).
 func (t *gitOpsTrait) gitConf(it *v1.Integration) v1.GitConfigSpec {

pkg/trait/gitops_test.go

@@ -26,6 +26,7 @@ import (
 	"time"
 
 	v1 "github.com/apache/camel-k/v2/pkg/apis/camel/v1"
+	"github.com/apache/camel-k/v2/pkg/internal"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 
 	"github.com/go-git/go-git/v5/plumbing/object"
@@ -187,3 +188,92 @@ func getRemoteURL(dirPath string) (string, error) {
 
 	return urls[0], nil
 }
+
+func newPipeGitOpsTraitTestSetup(t *testing.T) (*gitOpsTrait, v1.Pipe, v1.Integration) {
+	t.Helper()
+	trait, _ := newGitOpsTrait().(*gitOpsTrait)
+	trait.IntegrationDirectory = "integrations"
+
+	pipe := v1.NewPipe("default", "test-pipe")
+	pipe.UID = "pipe-uid-123"
+	pipe.Spec = v1.PipeSpec{
+		Source: v1.Endpoint{URI: ptr.To("timer:foo")},
+		Sink:   v1.Endpoint{URI: ptr.To("log:bar")},
+	}
+
+	it := v1.NewIntegration("default", "test-pipe")
+	it.OwnerReferences = []metav1.OwnerReference{
+		{
+			APIVersion: v1.SchemeGroupVersion.String(),
+			Kind:       v1.PipeKind,
+			Name:       pipe.Name,
+			UID:        pipe.UID,
+		},
+	}
+	now := metav1.Now().Rfc3339Copy()
+	it.Status = v1.IntegrationStatus{
+		Image:          "my-pipe-img",
+		BuildTimestamp: &now,
+	}
+
+	fakeClient, err := internal.NewFakeClient(&pipe)
+	require.NoError(t, err)
+	trait.Client = fakeClient
+
+	return trait, pipe, it
+}
+
+func TestGitOpsPushRepoPipe(t *testing.T) {
+	trait, pipe, it := newPipeGitOpsTraitTestSetup(t)
+	trait.Overlays = []string{"dev", "prod"}
+	srcGitDir := t.TempDir()
+	tmpGitDir := t.TempDir()
+	err := initFakeGitRepo(srcGitDir)
+	require.NoError(t, err)
+	trait.URL = srcGitDir
+
+	err = trait.pushGitOpsItInGitRepo(context.TODO(), &it, tmpGitDir, "fake")
+	require.NoError(t, err)
+
+	assert.Contains(t,
+		it.Status.GetCondition(v1.IntegrationConditionType("GitPushed")).Message,
+		"Pipe changes pushed to branch cicd/candidate-release",
+	)
+
+	// Verify branch and commit
+	lastCommitMessage, err := getLastCommitMessage(tmpGitDir)
+	require.NoError(t, err)
+	assert.Contains(t, lastCommitMessage, "feat(ci): build complete")
+	branchName, err := getBranchNameFromDir(tmpGitDir)
+	require.NoError(t, err)
+	assert.Contains(t, branchName, "cicd/candidate-release")
+
+	// Verify pipe.yaml in base (not integration.yaml)
+	_, err = os.Stat(filepath.Join(tmpGitDir, "integrations", pipe.Name, "base", "pipe.yaml"))
+	require.NoError(t, err)
+	_, err = os.Stat(filepath.Join(tmpGitDir, "integrations", pipe.Name, "base", "integration.yaml"))
+	assert.True(t, os.IsNotExist(err), "integration.yaml should not exist for Pipe gitops")
+
+	// Verify overlay directories with patch-pipe.yaml
+	for _, overlay := range []string{"dev", "prod"} {
+		overlayDir := filepath.Join(tmpGitDir, "integrations", pipe.Name, "overlays", overlay)
+		gitopsDir, err := os.Stat(overlayDir)
+		require.NoError(t, err)
+		assert.True(t, gitopsDir.IsDir())
+		_, err = os.Stat(filepath.Join(overlayDir, "patch-pipe.yaml"))
+		require.NoError(t, err, "patch-pipe.yaml should exist in overlay %s", overlay)
+		// Verify "all" profile is generated
+		allKust := filepath.Join(tmpGitDir, "integrations", "all", "overlays", overlay, "kustomization.yaml")
+		_, err = os.Stat(allKust)
+		require.NoError(t, err, "all profile kustomization.yaml should exist for overlay %s", overlay)
+	}
+}
+
+func TestGitOpsPipeRequiresURL(t *testing.T) {
+	trait, _, it := newPipeGitOpsTraitTestSetup(t)
+	// No URL configured, no it.Spec.Git fallback
+	tmpGitDir := t.TempDir()
+	err := trait.pushGitOpsItInGitRepo(context.TODO(), &it, tmpGitDir, "fake")
+	require.Error(t, err)
+	assert.Contains(t, err.Error(), "gitops trait requires a git URL when used with a Pipe")
+}

pkg/util/gitops/gitops.go

@@ -163,6 +163,10 @@ func EditPipe(kb *v1.Pipe, it *v1.Integration, kit *v1.IntegrationKit, toNamespa
 		traits.Container = &traitv1.ContainerTrait{}
 	}
 	traits.Container.Image = contImage
+	// We make sure not to propagate further the gitops trait
+	// to avoid infinite loops. If the user wants to do a chain based
+	// strategy, she can use the patch-pipe and continue the chain on purpose
+	traits.GitOps = nil
 	if kit != nil {
 		// We must provide the classpath expected for the IntegrationKit. This is calculated dynamically and
 		// would get lost when creating the non managed build Integration. For this reason