main
1<!DOCTYPE html>
2<html lang="en">
3<head>
4<!-- Sep 03, 2024 -->
5<meta charset="utf-8" />
6<meta name="viewport" content="width=device-width, initial-scale=1" />
7<title>Tekton Pipeline : another world without PipelineResources</title>
8<meta name="author" content="Vincent Demeester" />
9<meta name="generator" content="Org Mode" />
10<link rel='icon' type='image/x-icon' href='/images/favicon.ico'/>
11<meta name='viewport' content='width=device-width, initial-scale=1'>
12<link rel='stylesheet' href='/css/new.css' type='text/css'/>
13<link rel='stylesheet' href='/css/syntax.css' type='text/css'/>
14<link href='/index.xml' rel='alternate' type='application/rss+xml' title='Vincent Demeester' />
15</head>
16<body>
17<main id="content" class="content">
18<header>
19<h1 class="title">Tekton Pipeline : another world without PipelineResources</h1>
20</header><p>
21This document is a refreshed version of <a href="https://docs.google.com/document/d/1u6qO7CPtDnTOZMYFQ5ARysSOy8lfFeVw83C_5BxAKsw/edit#heading=h.yc5nzf2ze0dr">A world without PipelineResource</a> (only accessible
22if you are a member of the tekton community) as of <code>tekton/pipeline</code> <code>v0.17.x</code>.
23</p>
24<section id="outline-container-Goal%28s%29" class="outline-2">
25<h2 id="Goal%28s%29"><span class="todo TODO">TODO</span> Goal(s)</h2>
26<div class="outline-text-2" id="text-Goal%28s%29">
27</div>
28<div id="outline-container-PipelineResources%20%22problems%22" class="outline-3">
29<h3 id="PipelineResources%20%22problems%22">PipelineResources “problems”</h3>
30<div class="outline-text-3" id="text-PipelineResources%20%22problems%22">
31</div>
32<div id="outline-container-PipelineResources%20extra%28s%29" class="outline-4">
33<h4 id="PipelineResources%20extra%28s%29">PipelineResources extra(s)</h4>
34<div class="outline-text-4" id="text-PipelineResources%20extra%28s%29">
35<p>
36Related issue is <a href="https://github.com/tektoncd/pipeline/issues/3518">#3518</a>, but there might be others. It is not currently possible to pass
37extra certificates to a <code>PipelineResource</code> generated container, making, for example a
38self-signed <code>https</code> git clone from using <code>PipelineResource</code> impossible. This may apply to
39additional “extra” that we would want to apply to <code>PipelineResource</code> that we can apply to
40<code>Task</code> (additional volumes, …).
41</p>
42</div>
43</div>
44</div>
45</section>
46<section id="outline-container-Examples" class="outline-2">
47<h2 id="Examples"><span class="todo TODO">TODO</span> Examples</h2>
48<div class="outline-text-2" id="text-Examples">
49<div class='drawer logbook'>
50<h6>Logbook</h6>
51nil</div>
52
53<p>
54The examples in the document are based on the “User stories” list of <a href="https://docs.google.com/document/d/1h9n0Lod0OiJ_sP2HK8Ms7N04aee5LW8xfz5yMGCFMIs/edit?ts=5f96a3e8#">Tekton Pipeline v1
55API</a> document.
56</p>
57
58<p>
59We are going to use the <a href="https://github.com/tektoncd/catalog">catalog</a> task as much as we can. We are also going to use tekton
60bundle, that will be available starting from <code>v0.18.0</code>.
61</p>
62</div>
63<div id="outline-container-Prerequisite" class="outline-3">
64<h3 id="Prerequisite"><span class="done DONE">DONE</span> Prerequisite</h3>
65<div class="outline-text-3" id="text-Prerequisite">
66<div class='drawer logbook'>
67<h6>Logbook</h6>
68<ul class="org-ul">
69<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 15:36]</span></span></li>
70</ul>
71</div>
72
73<p>
74Let’s bundle some base tasks into a tekton bundle to “ease” of use.
75</p>
76
77<ul class="org-ul">
78<li><p>
79<code>git</code>
80</p>
81
82<div class="org-src-container">
83<pre class="src src-bash">tkn-oci push docker.io/vdemeester/tekton-base-git:v0.1 task/git-clone/0.2/git-clone.yaml task/git-cli/0.1/git-cli.yaml task/git-rebase/0.1/git-rebase.yaml
84</pre>
85</div></li>
86</ul>
87
88
89<p>
90Let’s also create a <i>generic</i> PVC for getting source code, …
91</p>
92
93<div class="org-src-container">
94<pre class="src src-yaml">apiVersion: v1
95kind: PersistentVolumeClaim
96metadata:
97 name: testpvc
98spec:
99 accessModes: [ ReadWriteMany ]
100 storageClassName: standard
101 resources:
102 requests:
103 storage: 1Gi
104</pre>
105</div>
106</div>
107</div>
108<div id="outline-container-Standard%20Go%20Pipeline" class="outline-3">
109<h3 id="Standard%20Go%20Pipeline"><span class="done DONE">DONE</span> Standard Go Pipeline</h3>
110<div class="outline-text-3" id="text-Standard%20Go%20Pipeline">
111<div class='drawer logbook'>
112<h6>Logbook</h6>
113<ul class="org-ul">
114<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 15:34]</span></span></li>
115</ul>
116</div>
117
118<p>
119A simple go pipeline is doing the following:
120</p>
121<ul class="org-ul">
122<li><b>linting</b> using <code>golangci-lint</code> - <a href="https://raw.githubusercontent.com/tektoncd/catalog/master/task/golangci-lint/0.1/golangci-lint.yaml">golangci-lint</a></li>
123<li><b>build</b> using <code>go build</code> - <a href="https://raw.githubusercontent.com/tektoncd/catalog/master/task/golang-build/0.1/golang-build.yaml">golang-build</a></li>
124<li><b>testing</b> using <code>go test</code> - <a href="https://raw.githubusercontent.com/tektoncd/catalog/master/task/golang-test/0.1/golang-test.yaml">golang-test</a></li>
125</ul>
126
127<div class="org-src-container">
128<pre class="src src-bash">tkn-oci push docker.io/vdemeester/tekton-golang:v0.1 task/golangci-lint/0.1/golangci-lint.yaml task/golang-build/0.1/golang-build.yaml task/golang-test/0.1/golang-test.yaml
129</pre>
130</div>
131
132<div class="org-src-container">
133<pre class="src src-yaml">---
134apiVersion: tekton.dev/v1beta1
135kind: Pipeline
136metadata:
137 name: std-golang
138spec:
139 params:
140 - name: package
141#- name: url
142# default: https://$(params.package) # that doesn't work
143 - name: revision
144 default: ""
145 workspaces:
146 - name: ws
147 tasks:
148 - name: fetch-repository
149 taskRef:
150 name: git-clone
151 bundle: docker.io/vdemeester/tekton-base-git:v0.1
152 workspaces:
153 - name: output
154 workspace: ws
155 params:
156 - name: url
157 value: https://$(params.package)
158 - name: build
159 taskRef:
160 name: golang-build
161 bundle: docker.io/vdemeester/tekton-golang:v0.1
162 runAfter: [ fetch-repository ]
163 params:
164 - name: package
165 value: $(params.package)
166 workspaces:
167 - name: source
168 workspace: ws
169 - name: lint
170 taskRef:
171 name: golangci-lint
172 bundle: docker.io/vdemeester/tekton-golang:v0.1
173 runAfter: [ fetch-repository ]
174 params:
175 - name: package
176 value: $(params.package)
177 workspaces:
178 - name: source
179 workspace: ws
180 - name: test
181 taskRef:
182 name: golang-test
183 bundle: docker.io/vdemeester/tekton-golang:v0.1
184 runAfter: [ build, lint ]
185 params:
186 - name: package
187 value: $(params.package)
188 workspaces:
189 - name: source
190 workspace: ws
191</pre>
192</div>
193
194<div class="org-src-container">
195<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
196kind: PipelineRun
197metadata:
198 generateName: run-std-go-
199spec:
200 pipelineRef:
201 name: std-golang
202 params:
203 - name: package
204 value: github.com/tektoncd/pipeline
205 workspaces:
206 - name: ws
207 volumeClaimTemplate:
208 spec:
209 accessModes:
210 - ReadWriteMany
211 resources:
212 requests:
213 storage: 1Gi
214</pre>
215</div>
216
217<p>
218Note:
219</p>
220<ul class="org-ul">
221<li><code>bundle</code> is duplicated a lot (default bundle would reduce verbosity).</li>
222<li><code>params</code> and <code>workspaces</code> are duplicated in there.
223<i>Maybe we could be able to specify workspace to be available for all tasks</i></li>
224</ul>
225</div>
226</div>
227<div id="outline-container-Standard%20Java%20Pipeline%28s%29" class="outline-3">
228<h3 id="Standard%20Java%20Pipeline%28s%29"><span class="done DONE">DONE</span> Standard Java Pipeline(s)</h3>
229<div class="outline-text-3" id="text-Standard%20Java%20Pipeline%28s%29">
230<div class='drawer logbook'>
231<h6>Logbook</h6>
232<ul class="org-ul">
233<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 17:23]</span></span></li>
234</ul>
235</div>
236
237<div class="org-src-container">
238<pre class="src src-bash">tkn-oci push docker.io/vdemeester/tekton-java:v0.1 task/maven/0.2/maven.yaml task/jib-gradle/0.1/jib-gradle.yaml task/jib-maven/0.1/jib-maven.yaml
239</pre>
240</div>
241</div>
242<div id="outline-container-Standard%20Java%20Pipeline%28s%29--Prerequisite" class="outline-4">
243<h4 id="Standard%20Java%20Pipeline%28s%29--Prerequisite"><span class="done DONE">DONE</span> Prerequisite</h4>
244<div class="outline-text-4" id="text-Standard%20Java%20Pipeline%28s%29--Prerequisite">
245<div class='drawer logbook'>
246<h6>Logbook</h6>
247<ul class="org-ul">
248<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 16:27]</span></span></li>
249</ul>
250</div>
251
252<p>
253Let’s have a <code>nexus</code> server running…
254</p>
255
256<div class="org-src-container">
257<pre class="src src-yaml">---
258apiVersion: apps/v1
259kind: Deployment
260metadata:
261 labels:
262 app: nexus
263 app.kubernetes.io/instance: nexus
264 app.kubernetes.io/name: nexus
265 app.kubernetes.io/part-of: nexus
266 name: nexus
267spec:
268 replicas: 1
269 selector:
270 matchLabels:
271 app: nexus
272 template:
273 metadata:
274 labels:
275 app: nexus
276 spec:
277 containers:
278 - name: nexus
279 image: docker.io/sonatype/nexus3:3.16.2
280 env:
281 - name: CONTEXT_PATH
282 value: /
283 imagePullPolicy: IfNotPresent
284 ports:
285 - containerPort: 8081
286 protocol: TCP
287 livenessProbe:
288 exec:
289 command:
290 - echo
291 - ok
292 failureThreshold: 3
293 initialDelaySeconds: 30
294 periodSeconds: 10
295 successThreshold: 1
296 timeoutSeconds: 1
297 readinessProbe:
298 failureThreshold: 3
299 httpGet:
300 path: /
301 port: 8081
302 scheme: HTTP
303 initialDelaySeconds: 30
304 periodSeconds: 10
305 successThreshold: 1
306 timeoutSeconds: 1
307 resources:
308 limits:
309 memory: 4Gi
310 cpu: 2
311 requests:
312 memory: 512Mi
313 cpu: 200m
314 terminationMessagePath: /dev/termination-log
315 volumeMounts:
316 - mountPath: /nexus-data
317 name: nexus-data
318 volumes:
319 - name: nexus-data
320 persistentVolumeClaim:
321 claimName: nexus-pv
322---
323apiVersion: v1
324kind: Service
325metadata:
326 labels:
327 app: nexus
328 name: nexus
329spec:
330 ports:
331 - name: 8081-tcp
332 port: 8081
333 protocol: TCP
334 targetPort: 8081
335 selector:
336 app: nexus
337 sessionAffinity: None
338 type: ClusterIP
339# ---
340# apiVersion: v1
341# kind: Route
342# metadata:
343# labels:
344# app: nexus
345# name: nexus
346# spec:
347# port:
348# targetPort: 8081-tcp
349# to:
350# kind: Service
351# name: nexus
352# weight: 100
353---
354apiVersion: v1
355kind: PersistentVolumeClaim
356metadata:
357 labels:
358 app: nexus
359 name: nexus-pv
360spec:
361 accessModes:
362 - ReadWriteOnce
363 resources:
364 requests:
365 storage: 5Gi
366</pre>
367</div>
368
369<p>
370… a maven-repo PVC
371</p>
372
373<div class="org-src-container">
374<pre class="src src-yaml">apiVersion: v1
375kind: PersistentVolumeClaim
376metadata:
377 name: maven-repo-pvc
378spec:
379 resources:
380 requests:
381 storage: 5Gi
382 volumeMode: Filesystem
383 accessModes:
384 - ReadWriteMany
385</pre>
386</div>
387
388<p>
389and a maven settings configmap
390</p>
391
392<div class="org-src-container">
393<pre class="src src-yaml">apiVersion: v1
394kind: ConfigMap
395metadata:
396 name: custom-maven-settings
397data:
398 settings.xml: |
399 <?xml version="1.0" encoding="UTF-8"?>
400 <settings>
401 <servers>
402 <server>
403 <id>nexus</id>
404 <username>admin</username>
405 <password>admin123</password>
406 </server>
407 </servers>
408 <mirrors>
409 <mirror>
410 <id>nexus</id>
411 <name>nexus</name>
412 <url>http://nexus:8081/repository/maven-public/</url>
413 <mirrorOf>*</mirrorOf>
414 </mirror>
415 </mirrors>
416 </settings>
417</pre>
418</div>
419</div>
420</div>
421<div id="outline-container-Maven" class="outline-4">
422<h4 id="Maven"><span class="done DONE">DONE</span> Maven</h4>
423<div class="outline-text-4" id="text-Maven">
424<div class='drawer logbook'>
425<h6>Logbook</h6>
426<ul class="org-ul">
427<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 16:40]</span></span></li>
428</ul>
429</div>
430
431<p>
432A simple <code>maven</code> project pipeline that build, run test, packages and publish artifacts
433(jars) to a maven repository. <i>Note: it uses a maven cache (<code>.m2</code>)</i>.
434</p>
435
436<p>
437The pipeline…
438</p>
439
440<div class="org-src-container">
441<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
442kind: Pipeline
443metadata:
444 name: std-maven
445spec:
446 params:
447 - name: url
448 - name: revision
449 default: ""
450 workspaces:
451 - name: ws
452 - name: local-maven-repo
453 - name: maven-settings
454 optional: true
455 tasks:
456 - name: fetch-repository
457 taskRef:
458 name: git-clone
459 bundle: docker.io/vdemeester/tekton-base-git:v0.1
460 workspaces:
461 - name: output
462 workspace: ws
463 params:
464 - name: url
465 value: $(params.url)
466 - name: unit-tests
467 taskRef:
468 bundle: docker.io/vdemeester/tekton-java:v0.1
469 name: maven
470 runAfter:
471 - fetch-repository
472 workspaces:
473 - name: source
474 workspace: ws
475 - name: maven-repo
476 workspace: local-maven-repo
477 - name: maven-settings
478 workspace: maven-settings
479 params:
480 - name: GOALS
481 value: ["package"]
482 - name: release-app
483 taskRef:
484 bundle: docker.io/vdemeester/tekton-java:v0.1
485 name: maven
486 runAfter:
487 - unit-tests
488 workspaces:
489 - name: source
490 workspace: ws
491 - name: maven-repo
492 workspace: local-maven-repo
493 - name: maven-settings
494 workspace: maven-settings
495 params:
496 - name: GOALS
497 value:
498 - deploy
499 - -DskipTests=true
500 - -DaltDeploymentRepository=nexus::default::http://nexus:8081/repository/maven-releases/
501 - -DaltSnapshotDeploymentRepository=nexus::default::http://nexus:8081/repository/maven-snapshots/
502</pre>
503</div>
504
505<p>
506… and the pipeline run
507</p>
508
509<div class="org-src-container">
510<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
511kind: PipelineRun
512metadata:
513 generateName: run-std-maven-
514spec:
515 pipelineRef:
516 name: std-maven
517 params:
518 - name: url
519 value: https://github.com/spring-projects/spring-petclinic
520 workspaces:
521 - name: maven-settings
522 configMap:
523 name: custom-maven-settings
524 items:
525 - key: settings.xml
526 path: settings.xml
527 - name: local-maven-repo
528 persistentVolumeClaim:
529 claimName: maven-repo-pvc
530 - name: ws
531 volumeClaimTemplate:
532 spec:
533 accessModes:
534 - ReadWriteMany
535 resources:
536 requests:
537 storage: 1Gi
538</pre>
539</div>
540
541<p>
542Notes:
543</p>
544<ul class="org-ul">
545<li>Need <code>affinity-assistant</code> to be disabled (as of today)</li>
546<li><code>params</code> and <code>workspaces</code> are duplicated in there.
547<i>Maybe we could be able to specify workspace to be available for all tasks</i></li>
548</ul>
549</div>
550</div>
551<div id="outline-container-Gradle" class="outline-4">
552<h4 id="Gradle"><span class="done DONE">DONE</span> Gradle</h4>
553<div class="outline-text-4" id="text-Gradle">
554<div class='drawer logbook'>
555<h6>Logbook</h6>
556<ul class="org-ul">
557<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-02 Mon 16:56]</span></span></li>
558</ul>
559</div>
560
561<p>
562A simple <code>gradle</code> project pipeline that build, run test, packages and publish artifacts
563(jars) to a maven repository. <i>Note: it uses a maven cache (<code>.m2</code>)</i>. This is the same as above
564but using <code>gradle</code> instead of <code>maven</code>.
565</p>
566
567<div class="org-src-container">
568<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
569kind: Pipeline
570metadata:
571 name: std-gradle
572spec:
573 params:
574 - name: url
575 - name: revision
576 default: ""
577 workspaces:
578 - name: ws
579 - name: local-maven-repo
580 - name: maven-settings
581 optional: true
582 tasks:
583 - name: fetch-repository
584 taskRef:
585 name: git-clone
586 bundle: docker.io/vdemeester/tekton-base-git:v0.1
587 workspaces:
588 - name: output
589 workspace: ws
590 params:
591 - name: url
592 value: $(params.url)
593 - name: unit-tests
594 taskRef:
595 bundle: docker.io/vdemeester/tekton-java:v0.1
596 name: gradle
597 runAfter:
598 - fetch-repository
599 workspaces:
600 - name: source
601 workspace: ws
602 - name: maven-repo
603 workspace: local-maven-repo
604 - name: maven-settings
605 workspace: maven-settings
606 params:
607 - name: GOALS
608 value: ["build"]
609 # - name: release-app
610 # taskRef:
611 # bundle: docker.io/vdemeester/tekton-java:v0.1
612 # name: gradle
613 # runAfter:
614 # - unit-tests
615 # workspaces:
616 # - name: source
617 # workspace: ws
618 # - name: maven-repo
619 # workspace: local-maven-repo
620 # - name: maven-settings
621 # workspace: maven-settings
622 # params:
623 # - name: GOALS
624 # value:
625 # - upload
626 # - -DskipTests=true
627 # - -DaltDeploymentRepository=nexus::default::http://nexus:8081/repository/maven-releases/
628 # - -DaltSnapshotDeploymentRepository=nexus::default::http://nexus:8081/repository/maven-snapshots/q
629</pre>
630</div>
631
632<p>
633and the run…
634</p>
635
636<div class="org-src-container">
637<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
638kind: PipelineRun
639metadata:
640 generateName: run-std-gradle-
641spec:
642 pipelineRef:
643 name: std-gradle
644 params:
645 - name: url
646 value: https://github.com/spring-petclinic/spring-petclinic-kotlin
647 workspaces:
648 - name: maven-settings
649 configMap:
650 name: custom-maven-settings
651 items:
652 - key: settings.xml
653 path: settings.xml
654 - name: local-maven-repo
655 persistentVolumeClaim:
656 claimName: maven-repo-pvc
657 - name: ws
658 volumeClaimTemplate:
659 spec:
660 accessModes:
661 - ReadWriteMany
662 resources:
663 requests:
664 storage: 1Gi
665</pre>
666</div>
667</div>
668</div>
669</div>
670<div id="outline-container-A%20source-to-image%20Pipeline" class="outline-3">
671<h3 id="A%20source-to-image%20Pipeline"><span class="done DONE">DONE</span> A source-to-image Pipeline</h3>
672<div class="outline-text-3" id="text-A%20source-to-image%20Pipeline">
673<div class='drawer logbook'>
674<h6>Logbook</h6>
675<ul class="org-ul">
676<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-10 Tue 17:04]</span></span></li>
677</ul>
678</div>
679
680<p>
681A pipeline that takes a repository with a <code>Dockerfile</code>, builds and pushes an image from it,
682and deploy it to kubernetes (using deployment/services).
683</p>
684
685<p>
686Let’s first setup a registry
687</p>
688
689<div class="org-src-container">
690<pre class="src src-shell">TMD=$(mktemp -d)
691
692# Generate SSL Certificate
693openssl req -newkey rsa:4096 -nodes -sha256 -keyout "${TMD}"/ca.key -x509 -days 365 \
694 -out "${TMD}"/ca.crt -subj "/C=FR/ST=IDF/L=Paris/O=Tekton/OU=Catalog/CN=registry"
695
696# Create a configmap from these certs
697kubectl create -n "${tns}" configmap sslcert \
698 --from-file=ca.crt="${TMD}"/ca.crt --from-file=ca.key="${TMD}"/ca.key
699</pre>
700</div>
701
702<div class="org-src-container">
703<pre class="src src-yaml">---
704apiVersion: apps/v1
705kind: Deployment
706metadata:
707 name: registry
708spec:
709 selector:
710 matchLabels:
711 run: registry
712 replicas: 1
713 template:
714 metadata:
715 labels:
716 run: registry
717 spec:
718 containers:
719 - name: registry
720 image: docker.io/registry:2
721 ports:
722 - containerPort: 5000
723 volumeMounts:
724 - name: sslcert
725 mountPath: /certs
726 env:
727 - name: REGISTRY_HTTP_TLS_CERTIFICATE
728 value: "/certs/ca.crt"
729 - name: REGISTRY_HTTP_TLS_KEY
730 value: "/certs/ca.key"
731 - name: REGISTRY_HTTP_SECRET
732 value: "tekton"
733 volumes:
734 - name: sslcert
735 configMap:
736 defaultMode: 420
737 items:
738 - key: ca.crt
739 path: ca.crt
740 - key: ca.key
741 path: ca.key
742 name: sslcert
743---
744apiVersion: v1
745kind: Service
746metadata:
747 name: registry
748spec:
749 ports:
750 - port: 5000
751 selector:
752 run: registry
753</pre>
754</div>
755</div>
756<div id="outline-container-buildah" class="outline-4">
757<h4 id="buildah"><span class="done DONE">DONE</span> buildah</h4>
758<div class="outline-text-4" id="text-buildah">
759<div class='drawer logbook'>
760<h6>Logbook</h6>
761<ul class="org-ul">
762<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-09 Mon 16:57]</span></span></li>
763</ul>
764</div>
765
766<div class="org-src-container">
767<pre class="src src-yaml">---
768apiVersion: tekton.dev/v1beta1
769kind: Pipeline
770metadata:
771 name: std-source-to-image-buildah
772spec:
773 params:
774 - name: url
775 - name: revision
776 default: ""
777 - name: image
778 default: "localhost:5000/foo"
779 - name: pushimage
780 default: "localhost:5000/foo"
781 workspaces:
782 - name: ws
783 - name: sslcertdir
784 optional: true
785 tasks:
786 - name: fetch-repository
787 taskRef:
788 name: git-clone
789 #bundle: docker.io/vdemeester/tekton-base-git:v0.1
790 workspaces:
791 - name: output
792 workspace: ws
793 params:
794 - name: url
795 value: $(params.url)
796 - name: build-and-push
797 taskRef:
798 name: buildah
799 #bundle: docker.io/vdemeester/tekton-builders:v0.1
800 runAfter: [ fetch-repository ]
801 params:
802 - name: IMAGE
803 value: $(params.pushimage)
804 - name: TLSVERIFY
805 value: "false"
806 workspaces:
807 - name: source
808 workspace: ws
809 # - name: sslcertdir
810 # workspace: sslcertdir
811 - name: deploy
812 runAfter: [ build-and-push ]
813 params:
814 - name: reference
815 value: $(params.image)@$(tasks.build-and-push.results.IMAGE_DIGEST)
816 taskSpec:
817 params:
818 - name: reference
819 steps:
820 - image: gcr.io/cloud-builders/kubectl@sha256:8ab94be8b2b4f3d117f02d868b39540fddd225447abf4014f7ba4765cb39f753
821 script: |
822 cat <<EOF | kubectl apply -f -
823 apiVersion: apps/v1
824 kind: Deployment
825 metadata:
826 name: foo-app
827 spec:
828 selector:
829 matchLabels:
830 run: foo-app
831 replicas: 1
832 template:
833 metadata:
834 labels:
835 run: foo-app
836 spec:
837 containers:
838 - name: foo
839 image: $(params.reference)
840</pre>
841</div>
842
843<div class="org-src-container">
844<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
845kind: PipelineRun
846metadata:
847 generateName: run-std-source-to-image-buildah-
848spec:
849 pipelineRef:
850 name: std-source-to-image-buildah
851 params:
852 - name: url
853 value: https://github.com/lvthillo/python-flask-docker
854 - name: pushimage
855 value: sakhalin.home:5000/foo
856 workspaces:
857 - name: ws
858 volumeClaimTemplate:
859 spec:
860 accessModes:
861 - ReadWriteOnce
862 resources:
863 requests:
864 storage: 1Gi
865</pre>
866</div>
867
868<p>
869Notes:
870</p>
871<ul class="org-ul">
872<li><code>deploy</code> may need it’s own task definition in the catalog. <code>kubectl-deploy-pod</code> is one but
873didn’t work properly</li>
874<li>rest is smooth</li>
875</ul>
876</div>
877</div>
878<div id="outline-container-s2i%20%28no%20%3DDockerfile%3D%29" class="outline-4">
879<h4 id="s2i%20%28no%20%3DDockerfile%3D%29"><span class="done DONE">DONE</span> s2i (no <code>Dockerfile</code>)</h4>
880<div class="outline-text-4" id="text-s2i%20%28no%20%3DDockerfile%3D%29">
881<div class='drawer logbook'>
882<h6>Logbook</h6>
883<ul class="org-ul">
884<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-10 Tue 16:59]</span></span></li>
885</ul>
886</div>
887
888<div class="org-src-container">
889<pre class="src src-yaml">---
890apiVersion: tekton.dev/v1beta1
891kind: Pipeline
892metadata:
893 name: std-source-to-image-s2i
894spec:
895 params:
896 - name: url
897 - name: revision
898 default: ""
899 - name: image
900 default: "localhost:5000/foo"
901 - name: pushimage
902 default: "localhost:5000/foo"
903 workspaces:
904 - name: ws
905 - name: sslcertdir
906 optional: true
907 tasks:
908 - name: fetch-repository
909 taskRef:
910 name: git-clone
911 #bundle: docker.io/vdemeester/tekton-base-git:v0.1
912 workspaces:
913 - name: output
914 workspace: ws
915 params:
916 - name: url
917 value: $(params.url)
918 - name: build-and-push
919 taskRef:
920 name: s2i
921 #bundle: docker.io/vdemeester/tekton-builders:v0.1
922 runAfter: [ fetch-repository ]
923 params:
924 - name: BUILDER_IMAGE
925 value: docker.io/fabric8/s2i-java:latest-java11
926 - name: S2I_EXTRA_ARGS
927 value: "--image-scripts-url=image:///usr/local/s2i"
928 - name: IMAGE
929 value: $(params.pushimage)
930 - name: TLSVERIFY
931 value: "false"
932 workspaces:
933 - name: source
934 workspace: ws
935 # - name: sslcertdir
936 # workspace: sslcertdir
937 - name: deploy
938 runAfter: [ build-and-push ]
939 params:
940 - name: reference
941 value: $(params.image)@$(tasks.build-and-push.results.IMAGE_DIGEST)
942 taskSpec:
943 params:
944 - name: reference
945 steps:
946 - image: gcr.io/cloud-builders/kubectl@sha256:8ab94be8b2b4f3d117f02d868b39540fddd225447abf4014f7ba4765cb39f753
947 script: |
948 cat <<EOF | kubectl apply -f -
949 apiVersion: apps/v1
950 kind: Deployment
951 metadata:
952 name: foo-app
953 spec:
954 selector:
955 matchLabels:
956 run: foo-app
957 replicas: 1
958 template:
959 metadata:
960 labels:
961 run: foo-app
962 spec:
963 containers:
964 - name: foo
965 image: $(params.reference)
966</pre>
967</div>
968
969<div class="org-src-container">
970<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
971kind: PipelineRun
972metadata:
973 generateName: run-std-source-to-image-s2i-
974spec:
975 pipelineRef:
976 name: std-source-to-image-s2i
977 params:
978 - name: url
979 value: https://github.com/siamaksade/spring-petclinic
980 - name: pushimage
981 value: sakhalin.home:5000/foo
982 workspaces:
983 - name: ws
984 volumeClaimTemplate:
985 spec:
986 accessModes:
987 - ReadWriteOnce
988 resources:
989 requests:
990 storage: 1Gi
991</pre>
992</div>
993
994<p>
995Notes:
996</p>
997<ul class="org-ul">
998<li><code>s2i</code> shares a lot with <code>buildah</code> or any <code>Dockerfile</code> build tool.
999This may <b>show</b> the need to compose tasks from other tasks. Here we do <code>s2i …
1000 --as-dockerfile</code> and then we just need to build the <code>Dockerfile</code>. This could be 2 separate
1001tasks but it would make the pipeline less efficient.</li>
1002</ul>
1003</div>
1004</div>
1005</div>
1006<div id="outline-container-A%20source-to-image%20%22knative%22%20Pipeline" class="outline-3">
1007<h3 id="A%20source-to-image%20%22knative%22%20Pipeline"><span class="done DONE">DONE</span> A source-to-image “knative” Pipeline</h3>
1008<div class="outline-text-3" id="text-A%20source-to-image%20%22knative%22%20Pipeline">
1009<div class='drawer logbook'>
1010<h6>Logbook</h6>
1011<ul class="org-ul">
1012<li>State “DONE” from “TODO” <span class="timestamp-wrapper"><span class="timestamp">[2020-11-10 Tue 17:02]</span></span></li>
1013</ul>
1014</div>
1015
1016<p>
1017A pipeline that takes a repository with a <code>Dockerfile</code>, builds and pushes an image from it,
1018and deploy it to kubernetes using knative services.
1019</p>
1020
1021<div class="org-src-container">
1022<pre class="src src-yaml">---
1023apiVersion: tekton.dev/v1beta1
1024kind: Pipeline
1025metadata:
1026 name: std-source-to-image-buildah-kn
1027spec:
1028 params:
1029 - name: url
1030 - name: revision
1031 default: ""
1032 - name: image
1033 default: "localhost:5000/foo"
1034 - name: pushimage
1035 default: "localhost:5000/foo"
1036 workspaces:
1037 - name: ws
1038 - name: sslcertdir
1039 optional: true
1040 tasks:
1041 - name: fetch-repository
1042 taskRef:
1043 name: git-clone
1044 #bundle: docker.io/vdemeester/tekton-base-git:v0.1
1045 workspaces:
1046 - name: output
1047 workspace: ws
1048 params:
1049 - name: url
1050 value: $(params.url)
1051 - name: build-and-push
1052 taskRef:
1053 name: buildah
1054 #bundle: docker.io/vdemeester/tekton-builders:v0.1
1055 runAfter: [ fetch-repository ]
1056 params:
1057 - name: IMAGE
1058 value: $(params.pushimage)
1059 - name: TLSVERIFY
1060 value: "false"
1061 workspaces:
1062 - name: source
1063 workspace: ws
1064 # - name: sslcertdir
1065 # workspace: sslcertdir
1066 - name: kn-deploy
1067 runAfter: [ build-and-push ]
1068 taskref:
1069 name: kn
1070 params:
1071 - name: ARGS
1072 value:
1073 - "service"
1074 - "create"
1075 - "hello"
1076 - "--force"
1077 - "--image=$(params.image)@$(tasks.build-and-push.results.IMAGE_DIGEST)"
1078</pre>
1079</div>
1080
1081<div class="org-src-container">
1082<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
1083kind: PipelineRun
1084metadata:
1085 generateName: run-std-source-to-image-buildah-kn-
1086spec:
1087 pipelineRef:
1088 name: std-source-to-image-buildah-kn
1089 params:
1090 - name: url
1091 value: https://github.com/lvthillo/python-flask-docker
1092 - name: pushimage
1093 value: sakhalin.home:5000/foo
1094 serviceAccountName: kn-deployer-account
1095 workspaces:
1096 - name: ws
1097 volumeClaimTemplate:
1098 spec:
1099 accessModes:
1100 - ReadWriteOnce
1101 resources:
1102 requests:
1103 storage: 1Gi
1104</pre>
1105</div>
1106</div>
1107</div>
1108<div id="outline-container-A%20canary%20deployment%20pipeline%20%28not%20from%20sources%29" class="outline-3">
1109<h3 id="A%20canary%20deployment%20pipeline%20%28not%20from%20sources%29"><span class="todo TODO">TODO</span> A canary deployment pipeline (not from sources)</h3>
1110</div>
1111
1112
1113
1114<div id="outline-container-A%20canary%20deployment%20pipeline%20%28iter8%29" class="outline-3">
1115<h3 id="A%20canary%20deployment%20pipeline%20%28iter8%29"><span class="todo TODO">TODO</span> A canary deployment pipeline (iter8)</h3>
1116<div class="outline-text-3" id="text-A%20canary%20deployment%20pipeline%20%28iter8%29">
1117<p>
1118This is taken from <a href="https://github.com/iter8-tools/canary-tekton-example">iter8 canary tekton example</a>.
1119</p>
1120
1121
1122<figure id="org2a3fc60">
1123<img src="./images/tekton/canary-pipeline.png" alt="canary-pipeline.png">
1124
1125</figure>
1126
1127<div class="org-src-container">
1128<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
1129kind: Task
1130metadata:
1131 name: identify-baseline-task
1132spec:
1133 description: |
1134 Identify the baseline deployment in a cluster namespace.
1135 params:
1136 - name: UID
1137 type: string
1138 default: "uid"
1139 description: |
1140 Unique identifier used to assocaite load with an experiment.
1141 Suitable values might be the experiment name of the task/pipeline run name/uid.
1142 - name: NAMESPACE
1143 type: string
1144 default: default
1145 description: The cluster namespace in which to search for the baseline.
1146 - name: EXPERIMENT_TEMPLATE
1147 type: string
1148 default: "experiment"
1149 description: Name of template that should be used for the experiment.
1150 workspaces:
1151 - name: source
1152 results:
1153 - name: baseline
1154 description: Name of the baseline deployment.
1155 steps:
1156 - name: update-experiment
1157 workingDir: $(workspaces.source.path)/$(params.UID)
1158 image: kalantar/yq-kubernetes
1159 script: |
1160 #!/usr/bin/env bash
1161 # Uncomment to debug
1162 set -x
1163
1164 # Identify baseline deployment for an experiment
1165 # This is heuristic; prefers to look at stable DestinationRule
1166 # But if this isn't defined will select first deployment that satisfies
1167 # the service selector (service from Experiment)
1168
1169 NAMESPACE=$(params.NAMESPACE)
1170 SERVICE=$(yq read $(params.EXPERIMENT_TEMPLATE) spec.service.name)
1171 ROUTER=$(yq read $(params.EXPERIMENT_TEMPLATE) spec.networking.id)
1172
1173 if [[ -z ${ROUTER} ]] || [[ "${ROUTER}" == "null" ]]; then
1174 ROUTER="${SERVICE}.${NAMESPACE}.svc.cluster.local"
1175 fi
1176
1177 echo "SERVICE=${SERVICE}"
1178 echo " ROUTER=${ROUTER}"
1179
1180 SUBSET=
1181 NUM_VS=$(kubectl --namespace ${NAMESPACE} get vs --selector=iter8-tools/router=${ROUTER} --output json | jq '.items | length')
1182 echo "NUM_VS=${NUM_VS}"
1183 if (( ${NUM_VS} > 0 )); then
1184 SUBSET=$(kubectl --namespace ${NAMESPACE} get vs --selector=iter8-tools/router=${ROUTER} --output json | jq -r '.items[0].spec.http[0].route[] | select(has("weight")) | select(.weight == 100) | .destination.subset')
1185 echo "SUBSET=$SUBSET"
1186 fi
1187
1188 DEPLOY_SELECTOR=""
1189 if [[ -n ${SUBSET} ]]; then
1190 NUM_DR=$(kubectl --namespace ${NAMESPACE} get dr --selector=iter8-tools/router=${ROUTER} --output json | jq '.items | length')
1191 echo "NUM_DR=${NUM_DR}"
1192 if (( ${NUM_DR} > 0 )); then
1193 DEPLOY_SELECTOR=$(kubectl --namespace ${NAMESPACE} get dr --selector=iter8-tools/router=${ROUTER} --output json | jq -r --arg SUBSET "$SUBSET" '.items[0].spec.subsets[] | select(.name == $SUBSET) | .labels | to_entries[] | "\(.key)=\(.value)"' | paste -sd',' -)
1194 fi
1195 fi
1196 echo "DEPLOY_SELECTOR=${DEPLOY_SELECTOR}"
1197
1198 if [ -z "${DEPLOY_SELECTOR}" ]; then
1199 # No stable DestinationRule found so find the deployment(s) implementing $SERVICE
1200 DEPLOY_SELECTOR=$(kubectl --namespace ${NAMESPACE} get service ${SERVICE} --output json | jq -r '.spec.selector | to_entries[] | "\(.key)=\(.value)"' | paste -sd',' -)
1201 fi
1202 echo "DEPLOY_SELECTOR=$DEPLOY_SELECTOR"
1203
1204 NUM_DEPLOY=$(kubectl --namespace ${NAMESPACE} get deployment --selector=${DEPLOY_SELECTOR} --output json | jq '.items | length')
1205 echo " NUM_DEPLOY=${NUM_DEPLOY}"
1206 BASELINE_DEPLOYMENT_NAME=
1207 if (( ${NUM_DEPLOY} > 0 )); then
1208 BASELINE_DEPLOYMENT_NAME=$(kubectl --namespace ${NAMESPACE} get deployment --selector=${DEPLOY_SELECTOR} --output jsonpath='{.items[0].metadata.name}')
1209 fi
1210 echo -n "${BASELINE_DEPLOYMENT_NAME}" | tee $(results.baseline.path)
1211---
1212apiVersion: tekton.dev/v1beta1
1213kind: Task
1214metadata:
1215 name: define-experiment-task
1216spec:
1217 description: |
1218 Define an iter8 canary Experiment from a template.
1219 workspaces:
1220 - name: source
1221 description: Consisting of kubernetes manifest templates (ie, the Experiment)
1222 params:
1223 - name: UID
1224 default: "uid"
1225 description: |
1226 Unique identifier used to assocaite load with an experiment.
1227 Suitable values might be the experiment name of the task/pipeline run name/uid.
1228 - name: EXPERIMENT_TEMPLATE
1229 type: string
1230 default: "experiment.yaml"
1231 description: An experiment resource that can be modified.
1232 - name: NAME
1233 type: string
1234 default: ""
1235 description: The name of the experiment resource to create
1236 - name: BASELINE
1237 type: string
1238 default: ""
1239 description: The name of the baseline resource
1240 - name: CANDIDATE
1241 type: string
1242 default: ""
1243 description: The name of the candidate (canary) resource
1244 results:
1245 - name: experiment
1246 description: Path to experiment (in workspace )
1247 steps:
1248 - name: update-experiment
1249 image: kalantar/yq-kubernetes
1250 workingDir: $(workspaces.source.path)/$(params.UID)
1251 script: |
1252 #!/usr/bin/env bash
1253
1254 OUTPUT="experiment-$(params.UID).yaml"
1255
1256 if [ -f "$(params.EXPERIMENT_TEMPLATE)" ]; then
1257 cp "$(params.EXPERIMENT_TEMPLATE)" "${OUTPUT}"
1258 else
1259 curl -s -o "${OUTPUT}" "$(params.EXPERIMENT_TEMPLATE)"
1260 fi
1261
1262 if [ ! -f "${OUTPUT}" ]; then
1263 echo "Can not read template: $(params.EXPERIMENT_TEMPLATE)"
1264 exit 1
1265 fi
1266
1267 # Update experiment template
1268 if [ "" != "$(params.NAME)" ]; then
1269 yq write --inplace "${OUTPUT}" metadata.name "$(params.NAME)"
1270 fi
1271 if [ "" != "$(params.BASELINE)" ]; then
1272 yq write --inplace "${OUTPUT}" spec.service.baseline "$(params.BASELINE)"
1273 fi
1274 if [ "" != "$(params.CANDIDATE)" ]; then
1275 yq write --inplace "${OUTPUT}" spec.service.candidates[0] "$(params.CANDIDATE)"
1276 fi
1277
1278 cat "${OUTPUT}"
1279 echo -n $(params.UID)/${OUTPUT} | tee $(results.experiment.path)
1280---
1281apiVersion: tekton.dev/v1beta1
1282kind: Task
1283metadata:
1284 name: apply-manifest-task
1285spec:
1286 description: |
1287 Create an iter8 canary Experiment from a template.
1288 workspaces:
1289 - name: manifest-dir
1290 description: Consisting of kubernetes manifests (ie, the Experiment)
1291 params:
1292 - name: MANIFEST
1293 type: string
1294 default: "manifest.yaml"
1295 description: The name of the file containing the kubernetes manifest to apply
1296 - name: TARGET_NAMESPACE
1297 type: string
1298 default: "default"
1299 description: The namespace in which the manifest should be applied
1300 steps:
1301 - name: apply-manifest
1302 image: kalantar/yq-kubernetes
1303 workingDir: $(workspaces.manifest-dir.path)
1304 script: |
1305 #!/usr/bin/env bash
1306
1307 # Create experiment in cluster
1308 kubectl --namespace $(params.TARGET_NAMESPACE) apply --filename "$(params.MANIFEST)"
1309---
1310apiVersion: tekton.dev/v1beta1
1311kind: Task
1312metadata:
1313 name: define-canary-task
1314spec:
1315 description: |
1316 Create YAML file needed to deploy the canary version of the application.
1317 Relies on kustomize and assumes a patch file template (PATCH_FILE) containing the keyword
1318 "VERSION" that can be replaced with the canary verion.
1319 params:
1320 - name: UID
1321 default: "uid"
1322 description: |
1323 Unique identifier used to assocaite load with an experiment.
1324 Suitable values might be the experiment name of the task/pipeline run name/uid.
1325 - name: image-repository
1326 description: Docker image repository
1327 default: ""
1328 - name: image-tag
1329 description: tag of image to deploy
1330 default: latest
1331 - name: PATCH_FILE
1332 default: kustomize/patch.yaml
1333 workspaces:
1334 - name: source
1335 results:
1336 - name: deployment-file
1337 description: Path to file (in workspace )
1338
1339 steps:
1340 - name: modify-patch
1341 image: alpine
1342 workingDir: $(workspaces.source.path)/$(params.UID)
1343 script: |
1344 #!/usr/bin/env sh
1345
1346 IMAGE_TAG=$(params.image-tag)
1347 PATCH_FILE=$(params.PATCH_FILE)
1348 IMAGE=$(params.image-repository):$(params.image-tag)
1349
1350 sed -i -e "s#iter8/reviews:istio-VERSION#${IMAGE}#" ${PATCH_FILE}
1351 sed -i -e "s#VERSION#${IMAGE_TAG}#g" ${PATCH_FILE}
1352 cat ${PATCH_FILE}
1353
1354 echo -n "deploy-$(params.UID).yaml" | tee $(results.deployment-file.path)
1355
1356 - name: generate-deployment
1357 image: smartive/kustomize
1358 workingDir: $(workspaces.source.path)/$(params.UID)
1359 command: [ "kustomize" ]
1360 args: [ "build", "kustomize", "-o", "deploy-$(params.UID).yaml" ]
1361
1362 - name: log-deployment
1363 image: alpine
1364 workingDir: $(workspaces.source.path)/$(params.UID)
1365 command: [ "cat" ]
1366 args: [ "deploy-$(params.UID).yaml" ]
1367---
1368apiVersion: tekton.dev/v1beta1
1369kind: Task
1370metadata:
1371 name: wait-completion-task
1372spec:
1373 description: |
1374 Wait until EXPERIMENT is completed;
1375 that is, condition ExperimentCompleted is true.
1376 params:
1377 - name: EXPERIMENT
1378 default: "experiment"
1379 description: Name of iter8 experiment
1380 - name: NAMESPACE
1381 default: default
1382 description: Namespace in which the iter8 experiment is defined.
1383 - name: TIMEOUT
1384 default: "1h"
1385 description: Amount of time to wait for experiment to complete.
1386 steps:
1387 - name: wait
1388 image: kalantar/yq-kubectl
1389 script: |
1390 #!/usr/bin/env sh
1391 set -x
1392
1393 kubectl --namespace $(params.NAMESPACE) wait \
1394 --for=condition=ExperimentCompleted \
1395 experiments.iter8.tools $(params.EXPERIMENT) \
1396 --timeout=$(params.TIMEOUT)
1397---
1398apiVersion: tekton.dev/v1beta1
1399kind: Task
1400metadata:
1401 name: cleanup-task
1402spec:
1403 workspaces:
1404 - name: workspace
1405 params:
1406 - name: UID
1407 default: "uid"
1408 description: |
1409 Unique identifier used to assocaite load with an experiment.
1410 Suitable values might be the experiment name of the task/pipeline run name/uid.
1411 steps:
1412 - name: clean-workspace
1413 image: alpine
1414 script: |
1415 #!/usr/bin/env sh
1416 set -x
1417
1418 rm -rf $(workspaces.workspace.path)/$(params.UID)
1419---
1420apiVersion: tekton.dev/v1beta1
1421kind: Task
1422metadata:
1423 name: identify-endpoint-task
1424spec:
1425 description: |
1426 Identify URL of application to be used buy load generator.
1427 params:
1428 - name: istio-namespace
1429 default: istio-system
1430 description: Namespace where Istio is installed.
1431 - name: application-query
1432 default: ""
1433 description: Application endpoint.
1434 results:
1435 - name: application-url
1436 description: The URL that can be used to apply load to the application.
1437 steps:
1438 - name: determine-server
1439 image: kalantar/yq-kubernetes
1440 script: |
1441 #!/usr/bin/env sh
1442
1443 # Determine the IP
1444 # Try loadbalancer on istio-ingressgateway
1445 IP=$(kubectl --namespace $(params.istio-namespace) get service istio-ingressgateway --output jsonpath='{.status.loadBalancer.ingress[0].ip}')
1446 # If not, try an external IP for a node
1447 echo "IP=${IP}"
1448 if [ -z "${IP}" ]; then
1449 IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type == "ExternalIP")].address}')
1450 fi
1451 echo "IP=${IP}"
1452 # If not, try an internal IP for a node (minikube)
1453 if [ -z "${IP}" ]; then
1454 IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type == "InternalIP")].address}')
1455 fi
1456 echo "IP=${IP}"
1457
1458 # Determine the port
1459 PORT=$(kubectl --namespace $(params.istio-namespace) get service istio-ingressgateway --output jsonpath="{.spec.ports[?(@.port==80)].nodePort}")
1460 echo "PORT=${PORT}"
1461
1462 HOST="${IP}:${PORT}"
1463 echo "HOST=$HOST"
1464
1465 echo -n "http://${HOST}/$(params.application-query)" | tee $(results.application-url.path)
1466---
1467apiVersion: tekton.dev/v1beta1
1468kind: Task
1469metadata:
1470 name: generate-load-task
1471spec:
1472 description: |
1473 Generate load by sending queries to URL every INTERVAL seconds.
1474 Load generation continues as long as the file terminate is not present.
1475 params:
1476 - name: UID
1477 default: "uid"
1478 description: |
1479 Unique identifier used to assocaite load with an experiment.
1480 Suitable values might be the experiment name of the task/pipeline run name/uid.
1481 - name: URL
1482 default: "http://localhost:8080"
1483 description: URL that should be used to generate load.
1484 - name: HOST
1485 default: ""
1486 description: Value to be added in Host header.
1487 - name: terminate
1488 default: ".terminate"
1489 description: Name of file that, if present, triggers termination of load generation.
1490 - name: INTERVAL
1491 default: "0.1"
1492 description: Interval (s) between generated requests.
1493 workspaces:
1494 - name: scratch
1495 steps:
1496 - name: generate-load
1497 image: kalantar/yq-kubernetes
1498 workingDir: $(workspaces.scratch.path)
1499 script: |
1500 #!/usr/bin/env bash
1501
1502 # Remove terminatation file if it exists (it should not)
1503 rm -f $(params.UID)/$(params.terminate) || true
1504
1505 echo "param HOST=$(params.HOST)"
1506 echo "param URL=$(params.URL)"
1507
1508 if [ "$(params.HOST)" == "" ]; then
1509 HOST=
1510 elif [ "$(params.HOST)" == "\*" ]; then
1511 HOST=
1512 else
1513 HOST=$(params.HOST)
1514 fi
1515 echo "computed HOST=$HOST"
1516
1517 # Optionally use a Host header in requests
1518 if [ -z ${HOST} ]; then
1519 echo "curl -o /dev/null -s -w \"%{http_code}\\n\" $(params.URL)"
1520 else
1521 echo "curl -H \"Host: ${HOST}\" -o /dev/null -s -w \"%{http_code}\\n\" $(params.URL)"
1522 fi
1523
1524 # Generate load until the file terminate is created.
1525 REQUESTS=0
1526 ERRORS=0
1527 while [ 1 ]; do
1528 if [ -f $(params.UID)/$(params.terminate) ]; then
1529 echo "Terminating load; ${REQUESTS} requests sent; ${ERRORS} had errors."
1530 break
1531 fi
1532 sleep $(params.INTERVAL)
1533 OUT=
1534 if [ -z ${HOST} ]; then
1535 OUT=$(curl -o /dev/null -s -w "%{http_code}\n" $(params.URL))
1536 else
1537 OUT=$(curl -H "Host: ${HOST}" -o /dev/null -s -w "%{http_code}\n" $(params.URL))
1538 fi
1539 if [ "${OUT}" != "200" ]; then ((ERRORS++)); echo "Not OK: ${OUT}"; fi
1540 ((REQUESTS++))
1541 done
1542---
1543apiVersion: tekton.dev/v1beta1
1544kind: Task
1545metadata:
1546 name: stop-load-task
1547spec:
1548 description: |
1549 Trigger the termination of experiment load.
1550 params:
1551 - name: UID
1552 default: "uid"
1553 description: |
1554 Unique identifier used to assocaite load with an experiment.
1555 Suitable values might be the experiment name of the task/pipeline run name/uid.
1556 - name: terminate
1557 default: ".terminate"
1558 description: Name of file that, if present, triggers termination of load generation.
1559 workspaces:
1560 - name: scratch
1561 steps:
1562 - name: wait
1563 image: alpine
1564 workingDir: $(workspaces.scratch.path)
1565 script: |
1566 #!/usr/bin/env sh
1567
1568 # To avoid conflicts, use a run specific subdirectory
1569 mkdir -p $(params.UID)
1570 touch $(params.UID)/$(params.terminate)
1571---
1572apiVersion: tekton.dev/v1beta1
1573kind: Task
1574metadata:
1575 name: queue-request-task
1576spec:
1577 description: |
1578 Place self at the end of a queue and wait until we are at the top.
1579 params:
1580 - name: UID
1581 default: "uid"
1582 description: |
1583 Unique identifier used to assocaite load with an experiment.
1584 Suitable values might be the experiment name of the task/pipeline run name/uid.
1585 - name: lock-dir
1586 default: ".lock"
1587 description: Name of directory to use to acquire mutex.
1588 - name: queue
1589 default: ".queue"
1590 description: Name of the file containing execution queue.
1591 - name: wait-time
1592 default: "20"
1593 description: Sleep time between attempts to aquire the lock.
1594 workspaces:
1595 - name: scratch
1596 steps:
1597 - name: queue
1598 image: alpine
1599 workingDir: $(workspaces.scratch.path)
1600 script: |
1601 #!/usr/bin/env sh
1602
1603 while [ "$(params.UID)" != "$(tail -n 1 $(params.queue))" ]; do
1604 if mkdir "$(params.lock-dir)"; then
1605 echo "queuing $(params.UID)"
1606 echo $(params.UID) >> $(params.queue)
1607 rm -rf "$(params.lock-dir)"
1608 else
1609 sleep $(params.wait-time)
1610 fi
1611 done
1612 - name: wait-head
1613 image: alpine
1614 workingDir: $(workspaces.scratch.path)
1615 script: |
1616 #!/usr/bin/env sh
1617
1618 while [ "$(params.UID)" != "$(head -n 1 $(params.queue))" ]; do
1619 sleep $(params.wait-time)
1620 done
1621 echo "$(params.UID) proceeding"
1622---
1623apiVersion: tekton.dev/v1beta1
1624kind: Task
1625metadata:
1626 name: dequeue-request-task
1627spec:
1628 description: |
1629 Remove entry from top of queue.
1630 params:
1631 - name: queue
1632 default: ".queue"
1633 description: Name of the file containing execution queue.
1634 workspaces:
1635 - name: scratch
1636 steps:
1637 - name: dequeue
1638 image: alpine
1639 workingDir: $(workspaces.scratch.path)
1640 script: |
1641 #!/usr/bin/env sh
1642
1643 tail -n +2 $(params.queue) > /tmp/$$; mv /tmp/$$ $(params.queue)
1644</pre>
1645</div>
1646
1647<div class="org-src-container">
1648<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
1649kind: Pipeline
1650metadata:
1651 name: canary-rollout-iter8
1652spec:
1653 workspaces:
1654 - name: source
1655 - name: experiment-dir
1656 params:
1657 - name: application-source
1658 type: string
1659 description: URL of source git repository.
1660 default: ""
1661 - name: application-namespace
1662 type: string
1663 description: Target namespace for application.
1664 - name: application-query
1665 type: string
1666 description: Service query for load generation.
1667 default: ""
1668 - name: application-image
1669 type: string
1670 description: Docker image repository for image to deploy.
1671 - name: HOST
1672 type: string
1673 description: Value that should be sent in Host header in test queries
1674 default: ""
1675
1676 - name: experiment
1677 type: string
1678 description: Name of experiment to create.
1679 default: "experiment"
1680 - name: experiment-template
1681 type: string
1682 description: Template for experiment to create.
1683
1684 - name: terminate
1685 type: string
1686 default: ".terminate"
1687 description: Name of file that, if present, triggers termination of load generation.
1688
1689 tasks:
1690 - name: initialize-request
1691 taskRef:
1692 name: queue-request-task
1693 workspaces:
1694 - name: scratch
1695 workspace: experiment-dir
1696 params:
1697 - name: UID
1698 value: $(context.pipelineRun.uid)
1699
1700 - name: clone-source
1701 taskRef:
1702 name: git-clone
1703 runAfter:
1704 - initialize-request
1705 workspaces:
1706 - name: output
1707 workspace: source
1708 params:
1709 - name: url
1710 value: $(params.application-source)
1711 - name: revision
1712 value: master
1713 - name: deleteExisting
1714 value: "true"
1715 - name: subdirectory
1716 value: $(context.pipelineRun.uid)
1717
1718 - name: build-and-push-image
1719 taskRef:
1720 name: kaniko
1721 runAfter:
1722 - clone-source
1723 timeout: "15m"
1724 workspaces:
1725 - name: source
1726 workspace: source
1727 params:
1728 - name: DOCKERFILE
1729 value: ./$(context.pipelineRun.uid)/Dockerfile
1730 - name: CONTEXT
1731 value: ./$(context.pipelineRun.uid)
1732 - name: IMAGE
1733 value: $(params.application-image):$(tasks.clone-source.results.commit)
1734 - name: EXTRA_ARGS
1735 value: "--skip-tls-verify"
1736
1737 - name: identify-baseline
1738 taskRef:
1739 name: identify-baseline-task
1740 runAfter:
1741 - clone-source
1742 workspaces:
1743 - name: source
1744 workspace: source
1745 params:
1746 - name: UID
1747 value: $(context.pipelineRun.uid)
1748 - name: NAMESPACE
1749 value: $(params.application-namespace)
1750 - name: EXPERIMENT_TEMPLATE
1751 value: $(params.experiment-template)
1752
1753 - name: define-experiment
1754 taskRef:
1755 name: define-experiment-task
1756 runAfter:
1757 - clone-source
1758 - identify-baseline
1759 workspaces:
1760 - name: source
1761 workspace: source
1762 params:
1763 - name: UID
1764 value: $(context.pipelineRun.uid)
1765 - name: EXPERIMENT_TEMPLATE
1766 value: $(params.experiment-template)
1767 - name: NAME
1768 value: $(context.pipelineRun.uid)
1769 - name: BASELINE
1770 value: $(tasks.identify-baseline.results.baseline)
1771 - name: CANDIDATE
1772 value: reviews-$(tasks.clone-source.results.commit)
1773
1774 - name: create-experiment
1775 taskRef:
1776 name: apply-manifest-task
1777 runAfter:
1778 - define-experiment
1779 workspaces:
1780 - name: manifest-dir
1781 workspace: source
1782 params:
1783 - name: TARGET_NAMESPACE
1784 value: $(params.application-namespace)
1785 - name: MANIFEST
1786 value: $(tasks.define-experiment.results.experiment)
1787
1788 - name: define-canary
1789 taskRef:
1790 name: define-canary-task
1791 runAfter:
1792 - clone-source
1793 workspaces:
1794 - name: source
1795 workspace: source
1796 params:
1797 - name: UID
1798 value: $(context.pipelineRun.uid)
1799 - name: image-repository
1800 value: $(params.application-image)
1801 - name: image-tag
1802 value: $(tasks.clone-source.results.commit)
1803
1804 - name: deploy-canary
1805 taskRef:
1806 name: apply-manifest-task
1807 runAfter:
1808 - create-experiment
1809 - build-and-push-image
1810 - define-canary
1811 workspaces:
1812 - name: manifest-dir
1813 workspace: source
1814 params:
1815 - name: TARGET_NAMESPACE
1816 value: $(params.application-namespace)
1817 - name: MANIFEST
1818 value: $(context.pipelineRun.uid)/$(tasks.define-canary.results.deployment-file)
1819
1820 - name: identify-endpoint
1821 taskRef:
1822 name: identify-endpoint-task
1823 runAfter:
1824 - initialize-request
1825 params:
1826 - name: application-query
1827 value: $(params.application-query)
1828
1829 - name: generate-load
1830 taskRef:
1831 name: generate-load-task
1832 runAfter:
1833 - create-experiment
1834 - identify-endpoint
1835 workspaces:
1836 - name: scratch
1837 workspace: experiment-dir
1838 params:
1839 - name: UID
1840 value: $(context.pipelineRun.uid)
1841 - name: URL
1842 value: $(tasks.identify-endpoint.results.application-url)
1843 - name: HOST
1844 value: $(params.HOST)
1845 - name: terminate
1846 value: $(params.terminate)
1847
1848 - name: wait-completion
1849 taskRef:
1850 name: wait-completion-task
1851 runAfter:
1852 - deploy-canary
1853 params:
1854 - name: EXPERIMENT
1855 value: $(context.pipelineRun.uid)
1856 - name: NAMESPACE
1857 value: $(params.application-namespace)
1858
1859 - name: stop-load-generation
1860 runAfter:
1861 - wait-completion
1862 taskRef:
1863 name: stop-load-task
1864 workspaces:
1865 - name: scratch
1866 workspace: experiment-dir
1867 params:
1868 - name: UID
1869 value: $(context.pipelineRun.uid)
1870 - name: terminate
1871 value: $(params.terminate)
1872
1873 finally:
1874 - name: cleanup-scratch-workspace
1875 taskRef:
1876 name: cleanup-task
1877 workspaces:
1878 - name: workspace
1879 workspace: experiment-dir
1880 params:
1881 - name: UID
1882 value: $(context.pipelineRun.uid)
1883 - name: cleanup-source-workspace
1884 taskRef:
1885 name: cleanup-task
1886 workspaces:
1887 - name: workspace
1888 workspace: source
1889 params:
1890 - name: UID
1891 value: $(context.pipelineRun.uid)
1892 - name: complete-request
1893 taskRef:
1894 name: dequeue-request-task
1895 workspaces:
1896 - name: scratch
1897 workspace: experiment-dir
1898</pre>
1899</div>
1900
1901<div class="org-src-container">
1902<pre class="src src-yaml">apiVersion: tekton.dev/v1beta1
1903kind: PipelineRun
1904metadata:
1905 name: canary-rollout
1906spec:
1907 pipelineRef:
1908 name: canary-rollout-iter8
1909 serviceAccountName: default
1910 workspaces:
1911 - name: source
1912 persistentVolumeClaim:
1913 claimName: source-storage
1914 - name: experiment-dir
1915 persistentVolumeClaim:
1916 claimName: experiment-storage
1917 params:
1918 - name: application-source
1919 value: https://github.com/kalantar/reviews
1920 - name: application-namespace
1921 value: bookinfo-iter8
1922 - name: application-image
1923 value: kalantar/reviews
1924 - name: application-query
1925 value: productpage
1926
1927 - name: HOST
1928 value: "bookinfo.example.com"
1929
1930 - name: experiment-template
1931 value: iter8/experiment.yaml
1932</pre>
1933</div>
1934</div>
1935</div>
1936<div id="outline-container-A%20canary%20%22knative%22%20deployment%20pipeline" class="outline-3">
1937<h3 id="A%20canary%20%22knative%22%20deployment%20pipeline"><span class="todo TODO">TODO</span> A canary “knative” deployment pipeline</h3>
1938</div>
1939
1940<div id="outline-container-A%20%22matrix%22%20build%20pipeline" class="outline-3">
1941<h3 id="A%20%22matrix%22%20build%20pipeline"><span class="todo TODO">TODO</span> A “matrix” build pipeline</h3>
1942</div>
1943
1944<div id="outline-container-%3Dtektoncd%2Fpipeline%3D%20project%20pipeline" class="outline-3">
1945<h3 id="%3Dtektoncd%2Fpipeline%3D%20project%20pipeline"><span class="todo TODO">TODO</span> <code>tektoncd/pipeline</code> project pipeline</h3>
1946</div>
1947
1948<div id="outline-container-Netlify%20flow" class="outline-3">
1949<h3 id="Netlify%20flow"><span class="todo TODO">TODO</span> Netlify flow</h3>
1950<div class="outline-text-3" id="text-Netlify%20flow">
1951<ul class="org-ul">
1952<li>Build and deploy a wip</li>
1953</ul>
1954</div>
1955</div>
1956</section>
1957<section id="outline-container-Issues" class="outline-2">
1958<h2 id="Issues"><span class="todo TODO">TODO</span> Issues</h2>
1959<div class="outline-text-2" id="text-Issues">
1960</div>
1961<div id="outline-container-No%20support%20for%20one-shot%20task%20with%20%3Dgit-clone%3D" class="outline-3">
1962<h3 id="No%20support%20for%20one-shot%20task%20with%20%3Dgit-clone%3D">No support for one-shot task with <code>git-clone</code></h3>
1963<div class="outline-text-3" id="text-No%20support%20for%20one-shot%20task%20with%20%3Dgit-clone%3D">
1964<p>
1965PipelineResource brought <i>pre</i> steps that would help running one task on top of a
1966GitResource for example. Let’s say you have a repository with a <code>Dockerfile</code>. All you want
1967is to build your <code>Dockerfile</code> in your CI. Without <code>PipelineResource</code> you are <i>stuck</i> to use a
1968<code>Pipeline</code>.
1969</p>
1970</div>
1971</div>
1972</section>
1973<section id="outline-container-Advantage" class="outline-2">
1974<h2 id="Advantage"><span class="todo TODO">TODO</span> Advantage</h2>
1975</section>
1976
1977<section id="outline-container-Next%20steps" class="outline-2">
1978<h2 id="Next%20steps"><span class="todo TODO">TODO</span> Next steps</h2>
1979</section>
1980
1981<section id="outline-container-References" class="outline-2">
1982<h2 id="References"><span class="todo TODO">TODO</span> References</h2>
1983<div class="outline-text-2" id="text-References">
1984<ul class="org-ul">
1985<li><a href="https://github.com/redhat-gpte-devopsautomation/app-dev-openshift-pipeline">https://github.com/redhat-gpte-devopsautomation/app-dev-openshift-pipeline</a></li>
1986<li><a href="https://gist.github.com/markito/9ef0329bce51a454e7ce5a0ed18a1eb1">https://gist.github.com/markito/9ef0329bce51a454e7ce5a0ed18a1eb1</a></li>
1987<li><a href="https://github.com/iter8-tools/canary-tekton-example">https://github.com/iter8-tools/canary-tekton-example</a></li>
1988<li><a href="https://github.com/ibm/ibm-garage-tekton-tasks">https://github.com/ibm/ibm-garage-tekton-tasks</a></li>
1989</ul>
1990</div>
1991</section>
1992</main>
1993<footer id="postamble" class="status">
1994<footer>
1995 <small><a href="/" rel="history">Index</a> • <a href="/sitemap.html">Sitemap</a> • <a href="https://dl.sbr.pm/">Files</a></small><br/>
1996 <small class='questions'>Questions, comments ? Please use my <a href="https://lists.sr.ht/~vdemeester/public-inbox">public inbox</a> by sending a plain-text email to <a href="mailto:~vdemeester/public-inbox@lists.sr.ht">~vdemeester/public-inbox@lists.sr.ht</a>.</small><br/>
1997 <small class='copyright'>
1998 Content and design by Vincent Demeester
1999 (<a rel='licence' href='http://creativecommons.org/licenses/by-nc-sa/3.0/'>Some rights reserved</a>)
2000 </small><br />
2001</footer>
2002</footer>
2003</body>
2004</html>