main
1<!DOCTYPE html>
2<html lang="en">
3<head>
4<!-- Sep 03, 2024 -->
5<meta charset="utf-8" />
6<meta name="viewport" content="width=device-width, initial-scale=1" />
7<title>OpenShift on VM Bare metal</title>
8<meta name="author" content="Vincent Demeester" />
9<meta name="generator" content="Org Mode" />
10<link rel='icon' type='image/x-icon' href='/images/favicon.ico'/>
11<meta name='viewport' content='width=device-width, initial-scale=1'>
12<link rel='stylesheet' href='/css/new.css' type='text/css'/>
13<link rel='stylesheet' href='/css/syntax.css' type='text/css'/>
14<link href='/index.xml' rel='alternate' type='application/rss+xml' title='Vincent Demeester' />
15</head>
16<body>
17<main id="content" class="content">
18<header>
19<h1 class="title">OpenShift on VM Bare metal</h1>
20</header><p>
21Let’s try to install OpenShift 4 on bare metal, where bare metal is virtual machine
22managed outside of OpenShift reach. We are trying to follow the <a href="https://docs.openshift.com/container-platform/4.4/installing/installing_bare_metal/installing-bare-metal.html">OpenShift Install on Bare
23metal</a> official documentation as much as possible.
24</p>
25
26<nav id="table-of-contents" role="doc-toc">
27<h2>Table of Contents</h2>
28<div id="text-table-of-contents" role="doc-toc">
29<ul>
30<li><a href="#Prerequisites">Prerequisites</a></li>
31<li><a href="#Provisionning%20Machines">Provisionning Machines</a>
32<ul>
33<li><a href="#Bootstraper">Bootstraper</a></li>
34<li><a href="#Load%20balancer">Load balancer</a></li>
35<li><a href="#Master%20and%20worker%20nodes">Master and worker nodes</a></li>
36</ul>
37</li>
38<li><a href="#Links">Links</a></li>
39</ul>
40</div>
41</nav>
42<section id="outline-container-Prerequisites" class="outline-2">
43<h2 id="Prerequisites">Prerequisites</h2>
44<div class="outline-text-2" id="text-Prerequisites">
45<p>
46In my <a href="infrastructure.html">infrastructure</a> I do have at my disposal 2 machines that both have libvirt (qemu+kvm)
47enabled (with nested virtualization support even 😝).
48</p>
49
50<ul class="org-ul">
51<li>okinawa: 8 cores, 32GiB memory</li>
52<li>wakasu: 8 cores, 64GiB memory</li>
53</ul>
54
55<p>
56Because we are going to install the machine <i>bare metal</i> on virtual machine, some
57assumptions are made (see <a href="infrastructure.html">Infrastructure</a> on those).
58</p>
59
60<ul class="org-ul">
61<li>VMs are created using a pre-defined Mac address, and the DHCP server on the network
62assign them a static IP (based on the mac address).</li>
63<li>This allows to create <span class="underline">ahead</span> of time DNS entries on the network so that the bootstrap
64and the cluster can find its own by domain name.</li>
65</ul>
66
67<p>
68My <i>home</i> network is using <code>home</code> as domain name tld. So VMs will be <code>vm1.home</code>, … and for
69the ocp cluster, <i>aliased</i> with <code>ocp.home</code>. We will have 3 master and 5 workers 🙃. The
70DNS entry looks more or less like
71</p>
72
73<div class="org-src-container">
74<pre class="src src-text">;; Load balancer
75vm0.home. IN A 192.168.1.120
76api.ocp.home. IN A 192.168.1.120
77api-int.ocp.home. IN A 192.168.1.120
78*.apps.ocp.home. IN A 192.168.1.120
79
80;; Masters
81vm1.home. IN A 192.168.1.121
82master1.ocp.home. IN A 192.168.1.121
83vm2.home. IN A 192.168.1.122
84master2.ocp.home. IN A 192.168.1.122
85vm3.home. IN A 192.168.1.123
86master3.ocp.home. IN A 192.168.1.123
87
88;; Workers
89vm4.home. IN A 192.168.1.124
90worker1.ocp.home. IN A 192.168.1.124
91vm5.home. IN A 192.168.1.125
92worker2.ocp.home. IN A 192.168.1.125
93vm6.home. IN A 192.168.1.126
94worker3.ocp.home. IN A 192.168.1.126
95vm7.home. IN A 192.168.1.127
96worker4.ocp.home. IN A 192.168.1.127
97vm8.home. IN A 192.168.1.128
98worker5.ocp.home. IN A 192.168.1.128
99
100;; Bootstrap machine
101vm9.home. IN A 192.168.1.129
102bootstrap.ocp.home. IN A 192.168.1.129
103
104;; etcd records
105etcd-0.ocp.home. IN A 192.168.1.121
106etcd-1.ocp.home. IN A 192.168.1.122
107etcd-2.ocp.home. IN A 192.168.1.123
108_etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-0.ocp.home.
109_etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-1.ocp.home.
110_etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-2.ocp.haome.
111</pre>
112</div>
113
114<p>
115As you can see, aside from the 3 masters and the 5 workers, we will have a load-balancer
116VM as documented in the official documentation.
117</p>
118
119<p>
120For reference, VMs Mac address compared to their IP.
121</p>
122
123<table>
124
125
126<colgroup>
127<col class="org-left">
128
129<col class="org-right">
130
131<col class="org-left">
132</colgroup>
133<tbody>
134<tr>
135<td class="org-left">MAC address</td>
136<td class="org-right">IP</td>
137<td class="org-left">Name</td>
138</tr>
139
140<tr>
141<td class="org-left">52:54:00:dd:a3:20</td>
142<td class="org-right">192.168.1.120</td>
143<td class="org-left">VM0</td>
144</tr>
145
146<tr>
147<td class="org-left">52:54:00:dd:a3:21</td>
148<td class="org-right">192.168.1.121</td>
149<td class="org-left">VM1</td>
150</tr>
151
152<tr>
153<td class="org-left">52:54:00:dd:a3:22</td>
154<td class="org-right">192.168.1.122</td>
155<td class="org-left">VM2</td>
156</tr>
157
158<tr>
159<td class="org-left">52:54:00:dd:a3:23</td>
160<td class="org-right">192.168.1.123</td>
161<td class="org-left">VM3</td>
162</tr>
163
164<tr>
165<td class="org-left">52:54:00:dd:a3:24</td>
166<td class="org-right">192.168.1.124</td>
167<td class="org-left">VM4</td>
168</tr>
169
170<tr>
171<td class="org-left">52:54:00:dd:a3:25</td>
172<td class="org-right">192.168.1.125</td>
173<td class="org-left">VM5</td>
174</tr>
175
176<tr>
177<td class="org-left">52:54:00:dd:a3:26</td>
178<td class="org-right">192.168.1.126</td>
179<td class="org-left">VM6</td>
180</tr>
181
182<tr>
183<td class="org-left">52:54:00:dd:a3:27</td>
184<td class="org-right">192.168.1.127</td>
185<td class="org-left">VM7</td>
186</tr>
187
188<tr>
189<td class="org-left">52:54:00:dd:a3:28</td>
190<td class="org-right">192.168.1.128</td>
191<td class="org-left">VM8</td>
192</tr>
193
194<tr>
195<td class="org-left">52:54:00:dd:a3:29</td>
196<td class="org-right">192.168.1.129</td>
197<td class="org-left">VM9</td>
198</tr>
199</tbody>
200</table>
201
202<p>
203Although I am tempted to use <a href="https://github.com/RedHatOfficial/ocp4-helpernode">ocp4-helpernode</a>, I am going to try to do as much as I can
204with my bare hands ✋.
205</p>
206</div>
207</section>
208<section id="outline-container-Provisionning%20Machines" class="outline-2">
209<h2 id="Provisionning%20Machines">Provisionning Machines</h2>
210<div class="outline-text-2" id="text-Provisionning%20Machines">
211<p>
212We will need to provision some machines, using <code>virsh</code>, <code>virt-install</code> or whatever works
213the best to be honest. This is also gonna be <b>heavily</b> based on
214<a href="https://github.com/RedHatOfficial/ocp4-helpernode/blob/master/docs/quickstart-static.md">ocp4-helpernode/quickstart-static.md</a>.
215</p>
216</div>
217<div id="outline-container-Bootstraper" class="outline-3">
218<h3 id="Bootstraper">Bootstraper</h3>
219<div class="outline-text-3" id="text-Bootstraper">
220<p>
221For the bootstrap VM (<code>vm9</code>), we can go with any system but, let’s follow closely the
222quistart and use Centos EL8. The boostraper machine will be on <code>wakasu</code> so we will need to
223target the libvirt daemon from there.
224</p>
225
226<div class="org-src-container">
227<pre class="src src-shell">export QEMU_URI=qemu+ssh://vincent@wakasu.home/system
228virt-install --connect=${QEMU_URI} \
229 --name="ocp4-bootstrap" --vcpus=4 --ram=8192 \
230 --disk path=/var/lib/libvirt/images/ocp-bootstrap.qcow2,bus=virtio,size=120 \
231 --boot menu=on --print-xml > ocp4-bootstrap.xml
232virsh --connect=${QEMU_URI} \
233 define --file ocp4-bootstrap.xml
234</pre>
235</div>
236</div>
237</div>
238<div id="outline-container-Load%20balancer" class="outline-3">
239<h3 id="Load%20balancer">Load balancer</h3>
240</div>
241
242<div id="outline-container-Master%20and%20worker%20nodes" class="outline-3">
243<h3 id="Master%20and%20worker%20nodes">Master and worker nodes</h3>
244<div class="outline-text-3" id="text-Master%20and%20worker%20nodes">
245<p>
246Let’s group machines between wakasu and okinawa. There will be 2 masters on wakasu and one
247on okinawa, 3 workers on wakasu, and 2 on okinawa.
248</p>
249
250<ul class="org-ul">
251<li><p>
252<code>wakasu</code>
253</p>
254<div class="org-src-container">
255<pre class="src src-bash">export QEMU_URI=qemu+ssh://vincent@wakasu.home/system
256virt-install --connect=${QEMU_URI} \
257 --name="ocp4-master1" --vcpus=4 --ram=12288 \
258 --disk path=/var/lib/libvirt/images/ocp4-master1.qcow2,bus=virtio,size=120 \
259 --os-variant rhel8.0 \
260 --network bridge=br1,mac.address=52:54:00:dd:a3:21 \
261 --boot menu=on --print-xml > ocp4-wakasu-master1.xml
262virt-install --connect=${QEMU_URI} \
263 --name="ocp4-master2" --vcpus=4 --ram=12288 \
264 --disk path=/var/lib/libvirt/images/ocp4-master2.qcow2,bus=virtio,size=120 \
265 --os-variant rhel8.0 \
266 --network bridge=br1,mac.address=52:54:00:dd:a3:22 \
267 --boot menu=on --print-xml > ocp4-wakasu-master2.xml
268virt-install --connect=${QEMU_URI} \
269 --name="ocp4-worker1" --vcpus=4 --ram=8192 \
270 --disk path=/var/lib/libvirt/images/ocp4-worker1.qcow2,bus=virtio,size=120 \
271 --os-variant rhel8.0 \
272 --network bridge=br1,mac.address=52:54:00:dd:a3:24 \
273 --boot menu=on --print-xml > ocp4-wakasu-worker1.xml
274virt-install --connect=${QEMU_URI} \
275 --name="ocp4-worker2" --vcpus=4 --ram=8192 \
276 --disk path=/var/lib/libvirt/images/ocp4-worker2.qcow2,bus=virtio,size=120 \
277 --os-variant rhel8.0 \
278 --network bridge=br1,mac.address=52:54:00:dd:a3:25 \
279 --boot menu=on --print-xml > ocp4-wakasu-worker2.xml
280virt-install --connect=${QEMU_URI} \
281 --name="ocp4-worker3" --vcpus=4 --ram=8192 \
282 --disk path=/var/lib/libvirt/images/ocp4-worker3.qcow2,bus=virtio,size=120 \
283 --os-variant rhel8.0 \
284 --network bridge=br1,mac.address=52:54:00:dd:a3:26 \
285 --boot menu=on --print-xml > ocp4-wakasu-worker3.xml
286for f in ocp4-wakasu-*.xml; do
287 virsh --connect=${QEMU_URI} define \
288 --file $f
289done
290</pre>
291</div>
292
293<p>
294Domain ocp4-master1 defined from ocp4-wakasu-master1.xml
295</p>
296
297<p>
298Domain ocp4-master2 defined from ocp4-wakasu-master2.xml
299</p>
300
301<p>
302Domain ocp4-worker1 defined from ocp4-wakasu-worker1.xml
303</p>
304
305<p>
306Domain ocp4-worker2 defined from ocp4-wakasu-worker2.xml
307</p>
308
309<p>
310Domain ocp4-worker3 defined from ocp4-wakasu-worker3.xml
311</p></li>
312</ul>
313
314
315
316<ul class="org-ul">
317<li><p>
318<code>okinawa</code>
319</p>
320<div class="org-src-container">
321<pre class="src src-bash">export QEMU_URI=qemu+ssh://vincent@okinawa.home/system
322virt-install --connect=${QEMU_URI} \
323 --name="ocp4-master3" --vcpus=4 --ram=12288 \
324 --disk path=/var/lib/libvirt/images/ocp4-master3.qcow2,bus=virtio,size=120 \
325 --os-variant rhel8.0 \
326 --network bridge=br1,mac.address=52:54:00:dd:a3:23 \
327 --boot menu=on --print-xml > ocp4-okinawa-master1.xml
328virt-install --connect=${QEMU_URI} \
329 --name="ocp4-worker4" --vcpus=4 --ram=8192 \
330 --disk path=/var/lib/libvirt/images/ocp4-worker4.qcow2,bus=virtio,size=120 \
331 --os-variant rhel8.0 \
332 --network bridge=br1,mac.address=52:54:00:dd:a3:27 \
333 --boot menu=on --print-xml > ocp4-okinawa-worker1.xml
334virt-install --connect=${QEMU_URI} \
335 --name="ocp4-worker5" --vcpus=4 --ram=8192 \
336 --disk path=/var/lib/libvirt/images/ocp4-worker5.qcow2,bus=virtio,size=120 \
337 --os-variant rhel8.0 \
338 --network bridge=br1,mac.address=52:54:00:dd:a3:28 \
339 --boot menu=on --print-xml > ocp4-okinawa-worker2.xml
340for f in ocp4-okinawa-*.xml; do
341 virsh --connect=${QEMU_URI} define \
342 --file $f
343done
344
345</pre>
346</div>
347
348<table>
349
350
351<colgroup>
352<col class="org-left">
353
354<col class="org-left">
355
356<col class="org-left">
357
358<col class="org-left">
359
360<col class="org-left">
361</colgroup>
362<tbody>
363<tr>
364<td class="org-left">Domain</td>
365<td class="org-left">ocp4-master3</td>
366<td class="org-left">defined</td>
367<td class="org-left">from</td>
368<td class="org-left">ocp4-okinawa-master1.xml</td>
369</tr>
370
371<tr>
372<td class="org-left">Domain</td>
373<td class="org-left">ocp4-worker4</td>
374<td class="org-left">defined</td>
375<td class="org-left">from</td>
376<td class="org-left">ocp4-okinawa-worker1.xml</td>
377</tr>
378
379<tr>
380<td class="org-left">Domain</td>
381<td class="org-left">ocp4-worker5</td>
382<td class="org-left">defined</td>
383<td class="org-left">from</td>
384<td class="org-left">ocp4-okinawa-worker2.xml</td>
385</tr>
386</tbody>
387</table></li>
388</ul>
389</div>
390</div>
391</section>
392<section id="outline-container-Links" class="outline-2">
393<h2 id="Links">Links</h2>
394<div class="outline-text-2" id="text-Links">
395<ul class="org-ul">
396<li><a href="https://github.com/RedHatOfficial/ocp4-helpernode">ocp4-helpernode</a></li>
397<li><a href="https://www.openshift.com/blog/openshift-4-bare-metal-install-quickstart">OpenShift 4 Bare metal quickstart</a> blog post</li>
398<li><a href="https://docs.openshift.com/container-platform/4.4/installing/installing_bare_metal/installing-bare-metal.html">OpenShift Install on Bare metal</a> official doc</li>
399</ul>
400</div>
401</section>
402</main>
403<footer id="postamble" class="status">
404<footer>
405 <small><a href="/" rel="history">Index</a> • <a href="/sitemap.html">Sitemap</a> • <a href="https://dl.sbr.pm/">Files</a></small><br/>
406 <small class='questions'>Questions, comments ? Please use my <a href="https://lists.sr.ht/~vdemeester/public-inbox">public inbox</a> by sending a plain-text email to <a href="mailto:~vdemeester/public-inbox@lists.sr.ht">~vdemeester/public-inbox@lists.sr.ht</a>.</small><br/>
407 <small class='copyright'>
408 Content and design by Vincent Demeester
409 (<a rel='licence' href='http://creativecommons.org/licenses/by-nc-sa/3.0/'>Some rights reserved</a>)
410 </small><br />
411</footer>
412</footer>
413</body>
414</html>