Open main menu

DAVE Developer's Wiki β

Changes

MISC-TN-005: Running AWS Greengrass Core on SBCSPG

10,004 bytes added, 14:29, 14 July 2021
Configuring AWS Greengrass on the cloud side and installing GGC on the edge device
{{InfoBoxTop}}
{{AppliesToSBCSPG}}
{{AppliesTo SBC Lynx TN}}
{{AppliesToIoT}}
{{AppliesTo IoT TN}}
{{InfoBoxBottom}}
{{WarningMessage|text=This technical note was validated against specific versions of hardware and software. It may not work with other versions.}}
In these days, we are witnessing the [https://www.itproportal.com/features/what-do-the-cloud-wars-mean-for-enterprises-in-2019/ cloud war]. Almost every week, the top three cloud providers⁠—Amazon, Microsoft, and Google⁠—add new services and new functionalities.
Technically speaking, it is hard to determine which platform is the best. Although basic services are similar, each platoform platform has different peculiarities that make it more or less suitable for a specific application. With regard to IoT applications, all major cloud platforms provide advanced services. Of course, (Industrial) IoT is a typical scenario in which DAVE Embedded Systems' products are used.
In principle, DAVE Embedded Systems' products are designed to be cloud-agnostic in order to let the users the freedom to choose the cloud platform they think best fits their needs.
This Technical Note, for instance, shows how to interface the SBCSPG Industrial IoT gateway to Amazon Web Services (AWS)by using Greengrass service. Specifically, this document shows how to configure and run the AWS Greengrass Core software (GGC) on the deviceSBCSPG. According to [https://aws.amazon.com/greengrass/faqs/ AWS website], AWS IoT Greengrass is  ''software that lets you run local compute, messaging, data caching, sync, and ML inference capabilities on connected devices in a secure way. With AWS IoT Greengrass, connected devices can run AWS Lambda functions, execute predictions based on machine learning models, keep device data in sync, and communicate with other devices securely – even when not connected to the Internet.'' ''AWS IoT Greengrass seamlessly extends AWS to devices so they can act locally on the data they generate, while still using the cloud for management, analytics, and durable storage. With AWS IoT Greengrass, you can use familiar languages and programming models to create your device software in the cloud, and then deploy it to your devices. AWS IoT Greengrass can be programmed to filter device data and only transmit necessary information back to the cloud.''  In other words, Greengrass allows you to implement edge computing functionalities easily and quickly by deploying them on the edge device through the AWS platform. The following is a short clip that illustrates well the concept of Greengrass.
''software that lets you run local compute, messaging, data caching, sync, and ML inference capabilities on connected devices in a secure way. With AWS IoT Greengrass, connected devices can run AWS Lambda functions, execute predictions based on machine learning models, keep device data in sync, and communicate with other devices securely – even when not connected to the Internet.
{{#ev:youtube|1rLxPOxVJoQ|400|center|Getting Started with AWS IoT Greengrass seamlessly extends AWS to devices so they can act locally on the data they generate, while still using the cloud for management, analytics, and durable storage. With AWS IoT Greengrass, you can use familiar languages and programming models to create your device software in the cloud, and then deploy it to your devices. AWS IoT Greengrass can be programmed to filter device data and only transmit necessary information back to the cloud.''|frame}}
In other words, GGC allows you to implement It is worth remembering that edge computing functionalities easily means ''a distributed computing paradigm which brings computation and quickly by deploying them on data storage closer to the edge device through the AWS platformlocation where it is needed, to improve response times and save bandwidth'', according to [https://en.wikipedia.org/wiki/Edge_computing Wikipedia].
It is worth remembering that edge computing means ''a distributed computing paradigm which brings computation and data storage closer to the location where it is needed, to improve response times and save bandwidth'', according to [https://en.wikipedia.org/wiki/Edge_computing Wikipedia]. Within the processing power limits of the edge device, moving such computations to the edge is can be an effective way to optimize costs. When the Internet connection is intermittent or poor, it can may even be even mandatory to meet system's requirements. Generally speaking, when implementing an IoT system, balancing between cloud computing and edge computing is one of the most important issues the system architect has to address.
Please note It should be stressed that this Technical Note is not a step-by-step guide to set up the edge device and the cloud platformto implement a Greengrass-based system. AWS documentation is rich and detailed in this regard. Rather, this document aims to underline some specific SBCSPG-related steps required to run GGC on this device successfully. The procedure is based on the [https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-gs.html ''Getting Started with AWS IoT Greengrass'' guide], which is, therefore, a recommended reading.
The test was performed with:
==Setting up the edge device==
The first thing to do was to rebuild the Linux kernel after adding the missing options⁠—if any⁠—required options⁠ required by GGC, which are detailed [https://docs.aws.amazon.com/en_us/greengrass/latest/developerguide/what-is-gg.html#gg-platforms here].
Once the device was up and running with the fresh Linux kernel, to enable IP routing to the Internet, the default gateway was setup like this:
</pre>
Then, the root file system has had to be configured according to the requirements described [https://docs.aws.amazon.com/en_us/greengrass/latest/developerguide/setup-filter.other.html here].
AWS documentation was a little bit confusing about the Node.JS and Java requirements. Both were installed, just in case. With regard to Java, OpenJDK 1.8.0 was installed.
To setup <code>cgroups</code>, [https://raw.githubusercontent.com/tianon/cgroupfs-mount/master/cgroupfs-mount this script] was used.
Finally, a couple of file system setting must had to be tuned, too. They are hardlink and softlink protections. For more details about thisthese settings, please refer to the following links:
*https://www.tecmint.com/protect-hard-and-symbolic-links-in-centos-rhel/
*https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_MRG/1.3/html/Realtime_Tuning_Guide/sect-Realtime_Tuning_Guide-General_System_Tuning-Setting_persistent_tuning_parameters.html
</pre>
To verify that the device is setup properly, it is recommended to use the dependency checker script provided by AWS itself. The following image shows the output of the checker executed on SBCSPG after completing the configuration.
As shown, only one check is highlighted, which refers to the kernel version. To run GGC 1.9.2, it is recommended a 4.4 kernel version or later. Although checker's report didn't warn about the use of containers, we verified that GGC didncouldn't work with default Lambda functions containerization flagson top of SBCSPG's 4.1.15 Linux kernel. Because a newer kernel was not available for SBCSPG at the time of this writing, to work around this issue, the [https://docs.aws.amazon.com/greengrass/latest/developerguide/lambda-group-config.html#lambda-containerization-groupsettings default Lamba functions containerization] was disabled.
In order to make the checker happy, two symbolic links were created as well:
</pre>
==Configuring AWS Greengrass Core on the cloud sideand installing GGC on the edge device==Module 2 of the ''Getting started guide'' details how to setup the Greengrass service on the cloud side.  Once the group is created, disable the default Lamba functions containerization, as stated previously. At the end of this step, a file has to be downloaded containing Core's security resources, which will be installed on the edge device. This archive also contains ''configuration information specific to your AWS IoT Greengrass core and the AWS IoT endpoint'', stored in the file <code>config.json</code>. After uncompressing it onto the SBCSPG, please verify that the <code>useSystemd</code> in <code>config.json</code> is set to ''yes''. If everything is configured properly, after staring Greengrass Core daemon, the following messages are logged into the <code>/greengrass/ggc/var/log/system/runtime.log</code> file:  <pre class="board-terminal">[2019-08-07T11:41:01.127+02:00][INFO]-===========================================[2019-08-07T11:41:01.127+02:00][INFO]-Greengrass Version: 1.9.2-RC4[2019-08-07T11:41:01.127+02:00][INFO]-Greengrass Root: /greengrass[2019-08-07T11:41:01.127+02:00][INFO]-Greengrass Write Directory: /greengrass/ggc[2019-08-07T11:41:01.127+02:00][INFO]-Group File Directory: /greengrass/ggc/deployment/group[2019-08-07T11:41:01.127+02:00][INFO]-Default Lambda UID: 109[2019-08-07T11:41:01.127+02:00][INFO]-Default Lambda GID: 114[2019-08-07T11:41:01.13+02:00][INFO]-===========================================[2019-08-07T11:41:01.131+02:00][INFO]-The current core is using the AWS IoT certificates with fingerprint. {"fingerprint": "3805f49f1af9e764b3ae8ca87f8ec43d7b94d1e3f05494fa166f6d3abbe14015"}[2019-08-07T11:41:01.133+02:00][INFO]-Will persist worker process info. {"dir": "/greengrass/ggc/packages/1.9.2/var/worker/processes"}[2019-08-07T11:41:01.135+02:00][INFO]-Will persist worker process info. {"dir": "/greengrass/ggc/packages/1.9.2/var/worker/processes"}[2019-08-07T11:41:01.14+02:00][INFO]-Subscription added to the system {"source": "arn:aws:lambda:us-east-2:504613037555:function:Greengrass_HelloWorld:1", "sourceType": 0, "subject": "hello/world", "target": "cloud", "targetType": 2}[2019-08-07T11:41:01.155+02:00][INFO]-The current core is using the AWS IoT certificates with fingerprint. {"fingerprint": "3805f49f1af9e764b3ae8ca87f8ec43d7b94d1e3f05494fa166f6d3abbe14015"}[2019-08-07T11:41:01.171+02:00][INFO]-No proxy URL found.[2019-08-07T11:41:01.213+02:00][INFO]-Started Deployment Agent to listen for updates.[2019-08-07T11:41:01.22+02:00][INFO]-Created worker. {"functionArn": "arn:aws:lambda:::function:GGDeviceCertificateManager", "workerId": "7c195254-cd25-4389-75d4-267b9cbd5bd7", "pid": 1600}[2019-08-07T11:41:01.224+02:00][INFO]-Connecting with MQTT. {"endpoint": "a15axy3tr9fuwq-ats.iot.us-east-2.amazonaws.com:8883", "clientId": "SBCSPG_grp0_Core"}[2019-08-07T11:41:01.495+02:00][WARN]-[5]GK Remote: Error retrieving public key data: ErrPrincipalNotConfigured: private key for MqttCertificate is not set[2019-08-07T11:41:01.595+02:00][INFO]-Created worker. {"functionArn": "arn:aws:lambda:::function:GGIPDetector:1", "workerId": "ff88e76a-9ca4-459f-7e20-689800240635", "pid": 1614}[2019-08-07T11:41:02.057+02:00][INFO]-Created worker. {"functionArn": "arn:aws:lambda:::function:GGConnManager", "workerId": "7e719557-11c0-4e47-4532-1d9eb06129fd", "pid": 1625}[2019-08-07T11:41:02.329+02:00][INFO]-Created worker. {"functionArn": "arn:aws:lambda:::function:GGCloudSpooler:1", "workerId": "8ae31447-0295-4ae8-450a-872305ce315b", "pid": 1636}[2019-08-07T11:41:02.338+02:00][WARN]-[5]GK Remote: Error retrieving public key data: ErrPrincipalNotConfigured: private key for MqttCertificate is not set[2019-08-07T11:41:02.428+02:00][INFO]-MQTT connection successful. {"attemptId": "uIzF", "clientId": "SBCSPG_grp0_Core"}[2019-08-07T11:41:02.428+02:00][INFO]-MQTT connection established. {"endpoint": "a15axy3tr9fuwq-ats.iot.us-east-2.amazonaws.com:8883", "clientId": "SBCSPG_grp0_Core"}[2019-08-07T11:41:02.429+02:00][INFO]-MQTT connection connected. Start subscribing. {"clientId": "SBCSPG_grp0_Core"}[2019-08-07T11:41:02.429+02:00][INFO]-Deployment agent connected to cloud.</pre>
==Deploying the test Lambda function to the edge device==
Module 3 (Part 1) of the ''Getting started guide'' shows hot to deploy a Lamba function⁠—which was created on the cloud⁠—to the edge device. This function, coded in Python, simply sends a "Hello, world!" message to the cloud at regular intervals.
 
==Testing the execution of the Lambda function on the edge device==
 
Once the function has been deployed, on the edge device the following messages appear in the <code>runtime.log</code> file.
<pre class="board-terminal">
[2019-08-07T11:41:02.43+02:00][INFO]-Start subscribing. {"numOfTopics": 2, "clientId": "SBCSPG_grp0_Core"}
[2019-08-07T11:41:02.43+02:00][INFO]-Trying to subscribe to topic $aws/things/SBCSPG_grp0_Core-gda/shadow/update/delta
[2019-08-07T11:41:02.6+02:00][INFO]-Trying to subscribe to topic $aws/things/SBCSPG_grp0_Core-gda/shadow/get/accepted
[2019-08-07T11:41:02.89+02:00][INFO]-Created worker. {"functionArn": "arn:aws:lambda:::function:GGTES", "workerId": "ed4e9532-696d-41a2-77e8-96f5226f8138", "pid": 1646}
[2019-08-07T11:41:02.933+02:00][INFO]-All topics subscribed. {"clientId": "SBCSPG_grp0_Core"}
[2019-08-07T11:41:03.306+02:00][INFO]-Created worker. {"functionArn": "arn:aws:lambda:::function:GGShadowService", "workerId": "d22b0689-28bc-472c-5637-dc67d673cb76", "pid": 1658}
[2019-08-07T11:41:03.708+02:00][INFO]-Created worker. {"functionArn": "arn:aws:lambda:::function:GGShadowSyncManager", "workerId": "9783bd15-4674-41ee-6578-653bfeef2cd3", "pid": 1668}
[2019-08-07T11:41:04.098+02:00][INFO]-Created worker. {"functionArn": "arn:aws:lambda:::function:GGSecretManager:1", "workerId": "d0ef6e38-e19c-4e1f-6e16-1a82ccd115fa", "pid": 1680}
[2019-08-07T11:41:04.328+02:00][INFO]-Started all system components.
[2019-08-07T11:41:04.37+02:00][INFO]-Created worker. {"functionArn": "arn:aws:lambda:us-east-2:504613037555:function:Greengrass_HelloWorld:1", "workerId": "373c16da-f79a-44b1-7640-7ca327d730ef", "pid": 1691}
[2019-08-07T11:41:05.963+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:11.004+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:16.036+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:21.074+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:26.116+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:31.151+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:36.186+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:41.23+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:46.278+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:51.322+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:41:56.359+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:01.399+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:06.441+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:11.481+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:16.522+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:21.57+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:26.612+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:31.656+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:36.705+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:41.742+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:46.787+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
[2019-08-07T11:42:51.828+02:00][INFO]-Handled functions request. {"functionName": "arn:aws:lambda:::function:GGRouter", "invocationId": ""}
</pre>
 
 
On the cloud, the log of the received messages looked like this:
==Testing the connection==
[[File:AWS-GGC-hello-world.png|thumb|center|600px|TBDReceived messages on the cloud]]
4,650
edits