MISC-TN-015: Yocto and git protocol error

Info Box Applies to AXEL Lite Technical Notes Applies to AXEL ULite Technical Notes Applies to ORCA Technical Notes Applies to MITO 8M Mini Technical Notes Applies to BORA Technical Notes Applies to BORA Xpress Technical Notes Applies to BORA Lite Technical Notes Applies to Yocto

History

Version	Date	Notes
1.0.0	Apr 2022	First public release

Introduction

Starting from beginning of 2022, there was a policy change on most git hosting services to restrict "unsecure" git protocol for security issues.

GitHub security access has been changed for accessing the git repositories via ssh: more information can be found in the following news Improving Git protocol security on GitHub.

As reported :

We’re changing which keys are supported in SSH and removing unencrypted Git protocol. Only users connecting via SSH or git:// will be affected. If your Git remotes start with https://, nothing in this post will affect you. If you’re an SSH user, read on for the details and timeline. 

Since March 15, 2022 unencrypted git protocol has been disabled:

This is already applied in DAVE BSPs published after March 2022 but older BSPs, published prior this change, may need to be fixed manually to use ssh instead of git:// to access some repositories

This Technical Note shows an example on how to update those older releases to solve this issue

BSP repo Manifest

DAVE Yocto BSPs uses repo to track the multiple layers required to setup the BSP itself. If git protocol is used to clone some these layers, user will encounter the above issue.

Here there is an example using the DESK-MX6-L-1.0.0 page instructions for building the overall BSP while fixing the manifest

Setup with original manifest

The original file, as per desk-mx6-l-1.0.1 tag, uses the git protocol to clone some layers:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>

  <default sync-j="2"/>

  <remote fetch="git://git.yoctoproject.org" name="yocto"/>
  <remote fetch="git://github.com/Freescale" name="freescale"/>
  <remote fetch="git://git.openembedded.org" name="oe"/>
  <remote fetch="git://github.com/OSSystems" name="OSSystems"/>
  <remote fetch="git://github.com/meta-qt5"  name="QT5"/>
  <remote fetch="git://github.com/meta-rust"  name="rust"/>
  <remote fetch="https://source.codeaurora.org/external/imx" name="CAF" />
  <remote fetch="ssh://git@git.dave.eu/" name="DAVE"/>
...
...

in this case, multiple access error will be prompted while running repo sync:

dvdk@vagrant:~/yocto$ ./repo sync
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0remote: Counting objects: 101, done.        
remote: Compressing objects: 100% (99/99), done.        
remote: Total 101 (delta 27), reused 0 (delta 0)        
Receiving objects: 100% (101/101), 15.88 KiB | 7.94 MiB/s, done.
Resolving deltas: 100% (27/27), done.
From ssh://git.dave.eu/desk-mx-l/desk-mx-l-bsp
 * [new branch]      hardknott             -> DAVE/hardknott
 * [new branch]      sumo                  -> DAVE/sumo
 * [new tag]         desk-mx6-l-1.0.1      -> desk-mx6-l-1.0.1
 * [new tag]         desk-mx6-l-1.0.0      -> desk-mx6-l-1.0.0
 * [new tag]         desk-mx6-l-1.0.2      -> desk-mx6-l-1.0.2
 * [new tag]         desk-mx6-l-3.0.0      -> desk-mx6-l-3.0.0
 * [new tag]         desk-mx6ul-l-1.0.0    -> desk-mx6ul-l-1.0.0
 * [new tag]         desk-mx6ul-l-1.0.1    -> desk-mx6ul-l-1.0.1
 * [new tag]         desk-mx6ul-l-3.0.0    -> desk-mx6ul-l-3.0.0
 * [new tag]         desk-mx8m-l-2.0.0-rc2 -> desk-mx8m-l-2.0.0-rc2
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
fatal: remote error: 
  The unauthenticated git protocol on port 9418 is no longer supported.
Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information.
100   145  100   145    0     0    147      0 --:--:-- --:--:-- --:--:--   147
100 15.5M  100 15.5M    0     0  2051k      0  0:00:07  0:00:07 --:--:-- 2660k
Receiving objects: 100% (182040/182040), 15.58 MiB | 37.81 MiB/s, done.
Resolving deltas: 100% (126131/126131), done.
From /home/dvdk/yocto/.repo/projects/sources/meta-fsl-bsp-release.git/clone.bundle
 * [new branch]      warrior-4.19.35-1.1.0     -> CAF/warrior-4.19.35-1.1.0
 * [new branch]      thud-4.19.35-1.0.0        -> CAF/thud-4.19.35-1.0.0
 * [new branch]      sumo-4.14.98-2.3.0        -> CAF/sumo-4.14.98-2.3.0
...
...

Fixing repo manifest

repo stores the current manifest in .repo/manifest.xml: this is the file that needs to be patched to solve this issue locally

After repo init and prior repo sync, user need to edit .repo/manifest.xml to change all occurrence of git:// protocol specifier to https://

This can be also applied with a simple sed command:

sed -i 's/git:\/\//https:\/\//g' .repo/manifest.xml

Please note that, for DESK-MX6-L-1.x.x, this is already fixed in desk-mx6-l-1.0.2 release:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>

  <default sync-j="2"/>

  <remote fetch="https://git.yoctoproject.org" name="yocto"/>
  <remote fetch="https://github.com/Freescale" name="freescale"/>
  <remote fetch="https://git.openembedded.org" name="oe"/>
  <remote fetch="https://github.com/OSSystems" name="OSSystems"/>
  <remote fetch="https://github.com/meta-qt5"  name="QT5"/>
  <remote fetch="https://github.com/meta-rust"  name="rust"/>
  <remote fetch="https://source.codeaurora.org/external/imx" name="CAF" />
  <remote fetch="ssh://git@git.dave.eu/" name="DAVE"/>
...
...

To be precise the diff between the two releases is:

diff --git a/default.xml b/default.xml
index f4d7443..1a4d7fb 100644
--- a/default.xml
+++ b/default.xml
@@ -3,12 +3,12 @@
 
   <default sync-j="2"/>
 
-  <remote fetch="git://git.yoctoproject.org" name="yocto"/>
-  <remote fetch="git://github.com/Freescale" name="freescale"/>
-  <remote fetch="git://git.openembedded.org" name="oe"/>
-  <remote fetch="git://github.com/OSSystems" name="OSSystems"/>
-  <remote fetch="git://github.com/meta-qt5"  name="QT5"/>
-  <remote fetch="git://github.com/meta-rust"  name="rust"/>
+  <remote fetch="https://git.yoctoproject.org" name="yocto"/>
+  <remote fetch="https://github.com/Freescale" name="freescale"/>
+  <remote fetch="https://git.openembedded.org" name="oe"/>
+  <remote fetch="https://github.com/OSSystems" name="OSSystems"/>
+  <remote fetch="https://github.com/meta-qt5"  name="QT5"/>
+  <remote fetch="https://github.com/meta-rust"  name="rust"/>
   <remote fetch="https://source.codeaurora.org/external/imx" name="CAF" />
   <remote fetch="ssh://git@git.dave.eu/" name="DAVE"/>

Layer recipes

While most of Yocto recipes used tar (or similar) archive to get source code, some of them download source code using git.

Usually https protocol is used for this, but in some cases SRC_URI may need to be updated if plain git protocol is used (and if the git server implements GitHub like security policies)

Workaround

As an workaround for the git://github.com/ access, it is possible to replace the git protocol to https using the git configuration command:

 git config --global url."https://github.com/".insteadOf git://github.com/

See more info here