Dietrich Schroff

Subscribe to Dietrich Schroff feed
Dietrich Schroff
Updated: 15 hours 50 min ago

Review@amazon: Microsoft Power Platform Enterprise Architecture

Wed, 2023-09-06 14:40

 This weekend i read the book "Microsoft Power Platform Enterprise Architecture" says about the book:

For forward-looking architects and decision makers who want to craft complex solutions to serve growing business needs, Microsoft Power Platform Enterprise Architecture offers an array of architectural best practices and techniques. With this book, you’ll learn how to design robust software using the tools available in the Power Platform suite and be able to integrate them seamlessly with various Microsoft 365 and Azure components. Unlike most other resources that are overwhelmingly long and unstructured, this book covers essential concepts using concise yet practical examples to help you save time.[...]

This is something i fully agree with. 

For more details please read my review at amazon :).

Postman: Scratchpad is end of life - Only cloud based workspaces possible --> How to keep the passwords/secrets secure?

Sun, 2023-07-30 05:00
Postman is the swiss army knife tool for dealing with HTTP(S) requests. You want to test anything with an GET, PUT, POST, OPTION call? Postman is the way to go.

But some months ago postman announced the following:

Scratchpad was the option to work locally on your PC with postman and nothing was synced to the cloud. That was really cool, because most of the HTTPS requests have to be authenticated in some way (basic auth, secrets for oauth, api-keys, etc...)
From my perspective cloud is a very cool thing, but moving all the sensitive parameters into the postman cloud is not really an option.
So what to do? 
  1. Check alternatives....
    But there is nothing like postman. e.g. the possibility to render the request to a curl, powershell, NodeJS axios, ... call and use this anywhere else....
  2. Remove all sensitive data from your request
    A good step (and just to note: it is bad design, if you ever stored secrets inside the calls :) )
  3. Now the calls do not work anymore....
But this problem was addressed really a long time ago (more than 4 years ago):

So here the link to the solution:

It is just this easy:use variables and do not fill the initial value!

Here the tests: 
First the local postman:

Second the browser postman:

As you can see: the current values are empty inside the cloud for all entries, which have the inital value not filled...
(By the way: the current value is always a copy of the inital value. If you put there something like test11 and test22, only test1 and test2 will show up in the cloud version).

LinkedIn: Microsoft 365 Backup for Dummies sponsored by Dummies

Sun, 2023-04-09 15:03

 This week Veeam published this booklet on for download:

The booklet contains 6 chapter - the last one is a summary "Six takeaways" like always in such "for dummies" books.

From my point of view chapter 1 & 2 can be skipped - this you should really know, if running M365 (motivation for M365 backup). 

Chapter 3 is about how the loss of files can be prevented with M365 mechanisms. This is about compliance center, retention policies and labels. But only the keywords are mentioned and no deeper insights are provided.

In chapter 4 many scenarios are described, how you can loose your data on M365. Here a quote:

I think this chapter can be skipped like chapter 1 & 2. 

Chapter 5 opens with a nice term which was new to me: BaaS - Backup as a Service. Never thought about this acronym. Completely clear, that some backups in cloud are done without having purchase storage or servers on premises. Nice thing inside this chapter: a checklist about data source, data properties and some others. Really nice.

Chapter 6 comes up with the takeaways. These are really worth reading.

LinkedIn: A Guide to Data Governance - Building a roadmap for trusted data

Mon, 2023-02-20 04:19

On linkedin from "The Cyber Security Hub" shared a nice booklet about data governance:

An like always: It is only a booklet with about 25 pages - so this is not really a deep dive into this topic, but it gives you a good overview and of course a good motivation:

These include the need to governdata to maintain its quality as well as the need to protect it. This entails the prerequisite need to discover data in your organization with cataloguing, scanning, and classifying your data to support this protection.

and if this is to abstract, you should consider the following use case (and i think this use case has to be considered):

However, for AI to become effective, the data it is using must be trusted. Otherwise decision accuracy may be compromised, decisions may be delayed, or actions missed which impacts on the bottom line. Companies do not want ‘garbage in, garbage out’.

The booklet contains the sections "Requirements for governing data in a modern enterprise", "components needed for data governance", "technology needed for end-to-end data governance" and "managing master data". All sections do not provide a walk through for achieving a good data governance, but there are many questions listed, which you should answer for your company and then move forward. 

If you already have a data governance in place: This book is a good challenge for your solution. And for sure you will find some points, which are missing :)

LinkedIn Topcs: Why dataverse is for everyone...

Sun, 2023-02-19 03:49

Today i got a notification from a Microsoft colleague about the following linkedin posting:

Some weeks ago i started with PowerApps - and there this "dataverse" was mentionend as well. 

If you walk through the presentation in this linkedin post, you get an idea what this dataverse can do. I found the following picture @Microsoft learn:

And there are more details, why and how dataverse can be used:

Sounds like a kind of datawarehouse centrally in the cloud. The most interesting point (like always): How to maintain this data, so that it is really usable...

Review: "Cloud Native Infrastructure with Azure" provided by Microsoft

Sun, 2023-02-12 14:33

Last week Microsoft published the following linkedin post:

On linkedin often tiny booklets are offered with around 10 up to 30 pages. But this offer from Microsoft is a book with 11 section and 289 pages.
If you are interested you can get it via this link (today this is still working, 12.2.2023):

If you are not convinced: Here the table of contents:

  1. Introduction: Why Cloud Native?
  2. Infrastructure as Code: Setting Up the Gateway
  3. Containerizing Your Application: More Than Boxes
  4. Kubernetes: The Grand Orchestrator
  5. Creating a Kubernetes Cluster in Azure
  6. Oberservability: Following the Breadcrumbs
  7. Service Discovery and Service Mesh: Finding New Territories and Crossing Borders
  8. Networking and Policy Management: Behold the Gatekeepers
  9. Distributed Databases and Storage: The Central Bank
  10. Getting the Message
  11. Serverless
  12. Conclusion

Sounds like many topics i want to read about... :)

Openssl: How to automate (without hitting the carriage return many times)

Sun, 2023-01-22 10:19

I think nearly everyone, who administers some PCs or servers has used openssl. And almost everything there is straight forward.

To create your own key and certificate, just run:

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
-keyout privateKey.key -out certificate.crt


You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:
But the problem with that approach:

You have to add the carriage returns for every line after the 5 dashes and then your certificate looks quite ugly (see red colored text):

openssl x509 -text -in certificate.crt -noout


        Version: 3 (0x2)
        Serial Number:
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = AU, ST = Some-State, O = Internet Widgits Pty Ltd
            Not Before: Jan 22 15:54:43 2023 GMT
            Not After : Jan 22 15:54:43 2024 GMT
        Subject: C = AU, ST = Some-State, O = Internet Widgits Pty Ltd
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
            X509v3 Authority Key Identifier:
            X509v3 Basic Constraints: critical
    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:

To avoid that you have just to expand your command with the following parameters:

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
-keyout privateKey.key -out certificate.crt \
-subj "/C=de/CN=schroff.special/OU=MyDepartment"

Typical attributes can be found here:

(Common Name, Organization, Organizational Unit, Country, Location)

FritzBox monitoring with telegraf, influx and grafana

Wed, 2022-11-09 15:01

On year ago i wrote about adding a FritzBox to my monitoring with grafana and influxdb:

This was done with collectd.

As i wrote in i upgraded my raspberry to 22.04 and along with many minor problems, collectd was gone. (and i think it will not be added anymore.)

All other monitorings use telegraf to get the data.

And there is a solution, which provides that:

You can follow the steps on this page. If you get no data - here is the commandline which you should use to test the connection:

python3 ./TelegrafFritzBox/ -p xxxxxx -i -u fritz8490

And this should be the command, which you use in 

/etc/telegraf/telegraf.d$ cat telegrafFritzBox.conf

The reward is really a very nice dashboard:


Installation and running RaceResult Presenter.exe on Ubuntu

Sat, 2022-11-05 12:54

For all who are active in sports and want to run a event with

In our case we have some Linux laptops which we want to use (Windows OS licenses missing).

RaceResult consists some some components

  • SEServer2.exe
  • Presenter.exe
  • CameraServer.exe
  • Transponder.exe
  • Moderator.exe
  • RRWS.exe

We want to run the Presenter.exe on a Linux box. So first step:

Install raceresult software with wine.

Preinstallation step:

winetricks vb6run

Installation step:

wine raceresult_12.4.25.0_386_stable.exe

(the exe is provided by raceresult)

Important thing: to run the Presenter.exe you need the SEServer2.exe running as well. Because RaceResult decided to let this server listen on port 1023, the installation has to be done twice. One with sudo and one without.

For the sudo installation please select all components. For the user installation only the presenter is needed (but you can go with all as well):

Then: start the SEServer 

sudo bash
cd /root/.wine/drive_c/Program Files (x86)/race result/race result 11/SEServer2
wine SEServer2.exe

Startup the presenter as well

cd ~/.wine/drive_c/Program Files (x86)/race result/race result 11/Presenter
wine Presenter.exe

Now the next tricky thing: You got a ses file from raceresult ( Copy this in both .wine directories to ./wine/drive_c and the open this ses file in the presenter:

Bam done. :)

Ubuntu raspberry pi: upgrade to 22.04...

Wed, 2022-09-28 14:13

Ubuntu released version 22.04 so i decided to make an update from

Ubuntu 21.04 (GNU/Linux 5.11.0-1027-raspi aarch64)


Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-1015-raspi aarch64)

But this was not so easy as i thought. Running on my raspberry pi the following services were running:

  • influxdb
  • collectd
  • telegraf
  • mosquitto
  • zigbee2mqtt
  • grafana

Without any problem only grafana was updated.

Collectd failed with this message:

Package 'collectd' has no installation candidate

no chance to fix that :(

With that i had to disable the collectd section in influxdb - with that session it failed with[2293]: run: open server: open service: Stat(): stat /usr/share/collectd/types.db: no such file or directory

Then zigbee2mqtt was not able to write to mosquitto. This is due a change of the default settings of mosquitto. allow_anonymous false is now default, so i had to add

allow_anonymous true

After that zigbee2mqtt was able to write data to mqtt again.

Last thing: restart of telegraf, because just did not start properly after the first reboot after the upgrade.

Not really a good update - my other raspberry pi will stay on 21.04 for some more months... To many request for german government portal...

Sun, 2022-07-10 13:55

In Germany, the property tax had to be reformed due to a ruling by the Federal Constitutional Court. So this year everybody got a mail with the request to register his property from july 1st up to the end of the year.

This should be done via ELSTER = electronic tax declaration (explanation of the acronym can be found here)  

I tried this yesterday and today, but the login page does not respond at all.

Today a message was posted on their website (orange box on the screenshot)

Restrictions when using the ElsterSmart app
It is currently not possible to use Mein ELSTER using the ElsterSmart app if the ElsterSmart app and Mein ELSTER are used on different devices (e.g. if you have installed ElsterSmart on your smartphone or tablet, but Mein ELSTER in the browser on your
use PC).


Availability Limitations
Due to the enormous interest in the forms for the property tax reform, there are currently restrictions on availability. We are already working intensively on being able to provide you with the usual quality as quickly as possible.

I think just at least 10 million citizens try to follow the request and the on datacenter the servers are on their limit:

Even the status page shows a 404:

So let's see if it is possible to follow the request until end of the year or if this deadline gets extended for some years :).


Review: Securing containers & cloud for dummies

Thu, 2022-05-26 04:26

Securing containers & cloud (provided by sysdig) is a booklet with 42 pages and 7 chapters. Like most of the "for dummies" series the last chapter is a summary with ten considerations.

But let's start from the beginning:
Chapter one "understanding cloud security" is a really nice abstract. Here some of the topic, which you should be aware of: "overprivileged identites", "visibility over cloud assets", "leaving out IT", "former employees, one-time users and guest accounts that are left active", ... With knowing that the following proposal is made: "to dectect and stop cyber threats [..] first step is to see them". Therefore a singe event store should be used and a open-source validation because of validation an transparency.
The second chapter is named "securing infrastructure as code (IaC). The typical arguments for IaC are speed, scalabilty, resilience, reproducibility but what about security? IaC is created by the developers and this code has to be checked as well as the application sources. And even if IaC is checked, configuration templates in  a CI/CD pipeline will suffer from drift. "Policy as code PaC allows you to leverage a shared policy model across multiple IaC, cloud, and Kubernetes environments.  Not only does PaC provide consistency and strengthen security, but also it saves time and allows you to scale faster."
"Preventing Vulnerabilites" is the third chapter. Many images in production contain patchable vulnerabilites, which should be patched. So the selecting of container images from every source (including DockerHub) without scanning them is not a good idea. One subsection here is "Automate vulnerability scanning in the CI/CD pipeline". I think this is something you should read in the booklet in detail.
After scanning for threats, the next chapter is about detecting and responding to threats. This chapter is only about 3 pages and it is more an appetizer for Falco, which is a solution from sysdig.
The sixth chapter is named "Targeting monitoring and troubleshooting issues" is is plea for open source. "Avoiding Vendor Lock-In" is key to success at least from the perspective of the authors.
As in the beginning mentioned the last chapter is a ten point summary of the topic. This is a fast checklist, you can use.

All in all a very good high level introduction into "Securing Containers & Cloud". I recommend all DevOps engineers and developers to spend half an hour to read this booklet.

Running a movie on an external DVD drive on a Chromebook (like HP x360)

Mon, 2021-12-27 02:56

In a first step this task sounds very easy:

  • watch a DVD on a chromebook


What are the problems?

  1. Using an external drive to access the dvd
  2. No appropriate app available in play store or chrome web store

There are different solutions out there. 

  1. Convert the DVD to a mp4 and watch this
  2. Use VLC from play store --> does not recognize the DVD
  3. Use VLC from chrome web store --> does not start at all
  4. Use linux development environment

Option 4 seemed to me as the most promising way to go.

Setting up linux is very easy:

 After that you have a debian bullseye running inside a container. Go to /etc/apt/sources.list and add "contrib" after "deb bullseye main " ("sudo bash" to get root). Then 

apt update
apt upgrade
apt install vlc libdvd-pkg
dpkg-reconfigure libdvd-pkg

After that vlc is configured including the libdvdcss for the DVD region codes.

One last problem is to access the DVD inside this linux container. This can be done via a double tap inside the file-manager on the chromebook and then you can choose inside the context menu "share with linux (Mit Linux teilen)".

This last step has to be done each time a DVD is inserted. 

So watching DVDs on a chromebook is not impossible, but it is not really user friendly...

influxdb: copying data with SELECT INTO - pay attention to the TAGS (or they are transformed to fields)

Sat, 2021-12-04 03:17

 If you are using influxdb, one usecase could be, copy the data from a measurement ("table") to another.

This can be done with this statement:

select * into testtable2 from testtable1

By the way: the CLI is opened with

/usr/bin/influx -unsafeSsl -ssl -database telegraf(if your database is named telegraf)

In my case (zigbee / mqtt / telegraf) the layout of mqtt_consumer measurement was like this:

> show tag keys from mqtt_consumer
name: mqtt_consumer
> show field keys from mqtt_consumer
name: mqtt_consumer
fieldKey    fieldType
--------    ---------
battery     float
contact     boolean
current     float
But after copying this to a testtable, the tags where gone and everything was a field. 

This is not a big problem - you can work with that data without a problem. BUT if you want to copy it back or merge it to the original table, you will get a table with the additional columns host_1 and topic_1.

This is because for influx you already had a column host. So it added a column field host_1. 

If a query in this new table (with host + host_1) spans over a time where both of this columns are in, you only select the data, with the entry host. If the time spans only entries with host_1, it is shown as host and you get your data. Really a unpredictable way to get data.

What is the solution? Easy:

select * into table1 from mqtt_consumer group by host,topic The "group by" does not group anything. It just tells influx: host & topic are tags and not fields. Please do not transform them...

Raspberry PI on Ubuntu: yarn: Cannot find module 'worker_threads'

Fri, 2021-11-26 12:54

This evening i tried to install a nodejs application with yarn on my raspberry pi. This failed with:

/usr/local/bin/yarn install
    throw err;
Error: Cannot find module 'worker_threads'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at /opt/zwavejs2mqtt/.yarn/releases/yarn-3.1.0-rc.8.cjs:287:2642
    at Object.<anonymous> (/opt/zwavejs2mqtt/.yarn/releases/yarn-3.1.0-rc.8.cjs:585:7786)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)

This error occurs because the nodejs version which is delivered by ubuntu is version v.10.19.0.

You have to download the armv8 package from

With version v16.13.0 the error was gone...


AZ-900 achieved: Microsoft Azure Fundamentals

Sat, 2021-11-20 02:30

Yesterday evening i passed Microsofts AZ-900 exam:

Taking the exam on site was no option because of COVID-19, so tried the first time the online option. Nice thing: Many schedules and i chose 20:45. 

As examinee you have to start your online session half an hour earlier and this time you really need for the onboarding: 

  1. Download the software to your PC and do some checks (audio, network, ...)
    This is an .exe - so only windows PCs are possible
  2. Install the app "Pearson VUE" on your smartphone to provide
    1. selfie
    2. passport/driver license/...
    3. photos of your room
  3. Talking to an instructor
    You are not allowed to wear a headset - even a watch is not allowed

 After that the exam is about 40 questions in 45 minutes - quite fair.

 The questions are about these topics:

  • Describe cloud concepts (20-25%)
  • Describe core Azure services (15-20%)
  • Describe core solutions and management tools on Azure (10-15%)
  • Describe general security and network security features (10-15%)
  • Describe identity, governance, privacy, and compliance features (15-20%)
  • Describe Azure cost management and Service Level Agreements (10-15%)

More information can be found here:

If you want to do this exam, start here:

Fritz!Box monitoring with grafana, influx, collectd and fritzcollectd

Sat, 2021-11-06 16:56

 A nice way to monitor your Fritz!Box is this here:

How can you achieve this:


Here a list of the software packages you have to install

apt install -y collectd python3-pip libxml2 libxml2-dev libxslt1-dev influxdb nodejs git make g++ gcc npm net-tools certbot mosquitto mosquitto-clients grafana-server

for grafana-server and influxdb you have to add new repositories, because they are still not included in ubuntu.

To tell collectd, that it shoud write to influxdb, you have to uncomment the following in collectd.conf:

<Plugin network>
        Server "localhost" "25826"

and in influxdb.conf:

  enabled = true
  bind-address = "localhost:25826"
  database = "collectd"
  retention-policy = ""
  typesdb = "/usr/share/collectd/types.db"
  parse-multivalue-plugin = "split"

and of course inside collectd.conf you have to add the fritzcollectd config from the github link above.

But with starting collectd you might get the error:

dlopen("/usr/lib/collectd/") failed: /usr/lib/collectd/ undefined symbol: PyFloat_Type

This can be solved with adding into /etc/default/collectd:


Zigbee: Setup zigbee2mqtt with usbstick conbee II & influxdb on a raspberry pi

Sat, 2021-11-06 11:20

Just a short walkthrough of all steps which are necessary:

1.) insert the usbstick and check if this special device is there: /dev/ttyACM0 

if this device is not showing up, it might be, that your kernel does not support usbserial. In my case i had to downgrade from ubuntu server 21.10 to 21.04.

2.) follow these steps:


apt-get install -y nodejs git make g++ gcc npm
git clone /opt/zigbee2mqtt
cd /opt/zigbee2mqtt

npm ci

if you get 

prebuild-install WARN install EACCES: permission denied, access '/root/.npm/_cacache'

then you should not use root for running this command.

cd /opt/zigbee2mqtt
chown -R ubuntu node_modules
rm node_modules/*
npm ci

3.) install mqtt 

apt install mosquitto mosquitto-clients

4.) add to /etc/mosquitto/mosqitto.conf the line

listener 1883

and restart mosquitto (systemctl restart mosquitto)


5.) then start the zigbee2mqtt:

cd /opt/zigbee2mqtt
npm start

 if you get

Zigbee2MQTT:error 2021-11-06 09:05:23: Error: Error while opening serialport 'Error: Error: No such device or address, cannot open /dev/ttyACM0' 

then you did not really check step 1.): please check that /dev/ttyACM0 is missing - if yes: for me the kernel module (to list: lsmod) usbserial was missing. It seems, that ubuntu missed that on 21.10 - so i reinstalled 21.04....

if you get

zigbee2MQTT:error 2021-11-06 14:54:11: MQTT failed to connect: connect ECONNREFUSED

 then you did not get mosquitto running. Check with systemctl status mosquitto and follow step 3 and 4.

6.) configure telegraf, so that the data from mosquitto is transferred to influxdb. So you have to add to telegraf.conf:


   servers = ["tcp://"]
   topics = [
   data_format = "json"

   urls = ["unix:///var/run/influxdb/influxdb.sock"]
   username = "admin"
   password = "XXXXX"

7.) add this user to influxdb:


influx -ssl -unsafeSsl (only influx if you have not enabled SSL)

create user admin with password 'XXXXXXX' with all privileges

8.) if you have joined a device this the zigbee2mqtt, then you have to give a friendy name inside /opt/zigbee2mqtt/data/configuration.yaml

   friendly_name: 'sensor/t1'

Review: Mastering Azure Machine Learning

Sat, 2021-10-30 12:51

Last week i stumbled upon this book and this weekend there was enough time to walk through it:


The book contains 14 chapters on 409 pages - but due to the layout, i think it can fit on 200 pages on a book with "default rendering".
The book is in addition divided in 4 sections: 1 - Azure Machine Learning / 2 - Experimentation and Data Preperation / 3 - Training Machine Learning Models / 4 Optimization and Deployment of Machine Learning Models

Chapter 1 is named "Building an end-to-end machine learning pipeline in Azure". I struggled with this title, but in the first section it is explained: "You can see it as an overview of the book". The subsections cover data exploration, data preparation, choosing the model, optimization and deploying/operating models. The chapter is a teaser with many graphs, examples, stragetgies - a fast end-to-end walk through.

"Choosing a machine learning service in Azure" is the title of the second chapter. Here is everything discussed about ML vs. AI and the Azure services, which provide these techniques (e.g. Data Science Virtual Machine, Azure Batch, Azure Databricks, Azure Functions, Azure IoT Edge, Custom Vision, Azure Machine Learning Designer, Machine Learning Studio,  ...). This chapter contains many screenshots and code snippets - from my point of view to much at this point.

In chapter three (Data experimentation and visualization using Azure) it is shown how to setup your environment via Azure CLI, so that you are able to perform these steps again and again for new projects. In addition it is presented how to run everything on the local machine and track the metrics and artifacts to the Azure workspace. After that visualization is explained including code examples. Pairplots, principal component analysis, quadratic discriminant analysis, stochastic neighbor embedding - Really cool.

Chapter 4 is about "ELT, data preparation and feature extraction". Here are some nice commands with Azure CLI provided: How to batch upload data up to the Azure storage accounts and attaching them to the ML workspace. And how to access this data via python.

Chapter 5 "Azure Machine Learning Pipelines" is about to make the content of chapter 4 reusable. I think nothing to note here - a nice reference for the python code which is needed.

"Advanced feature extraction with NLP" is chapter 6. NLP = natural language processing. Nothing more to say here.

The chaper 7 to 9 are about training machine learning models. I think i will not describe each of them. But here a short summary: It starts with decision trees as explanation and then does a deep dive in how to use LightGBM including the python code. Then the same for convolutional neural networks (CNN): explanation/motivation + coding. This is followed by the description of Azure Hyperdrive: tuning and optimizing the machine learning process. The concept of hyperparameters (e.g. number of neurons in a layer)  is introduced and how to choose them with grid sampling on an elastic cloud infrastructure. And last but not least: it is described how Azure provides "a service to users that automatically preprocesses your data, selects an ML model, and trains and optimizes the model to optimally fit your training data [...]".

Chapter 10 is about using clusters. This is a nice introduction about partitioning data, workloads and synchronizing worker nodes.
"Building a recommendation engine in Azure" is the title of chapter 11. Just some catchwords from the content: non-personalized, contentbased, rating-based, hybrid recommendations. After this chapter you will know, why amazons recommendations are like they are ;-)

In chapter 11 & 12 it is described, how to register, deploy and operate a recommendation engine or machine learning model up to MLOps.

The book closes with chapter 14 "What's next?". Most important point like everywhere: Automation...

Summary: I liked this book very much, because every topic starts with an excellent introduction and there are many code examples, so that you can us this book as reference as well. The basic understanding of the author is best described with the following quote:
"the most important tasks [are]: Data acquisition,  Data cleansing, Data labeling, Selecting an error metric. We don't want to blame anyone, but some machine learning engineers love to simply skip these topics and dive right into the fun parts, namely feature engineering, model selection, parameterization, and tuning." 

That hits the bull's eye.


Review: Intent based networking for dummies

Sat, 2021-09-11 16:38

I found the book intent-based networking on linkedin posted by juniper:

The book contains 5 chapters on 44 pages.

Chapter one (expressing intent and seeing the basics of IBN) tries to give a motivation for intent based networking. And the story goes like this: "humans are slow, expensive, error prone, and inconsistent. [...] the systems are vulerable to small mistakes that can have enormous costs to business."
In addition we have "inadequate automation", "data overload", and "stale documentation". (At this point i think we are generally doomed and should stop networking at all).
BUT with IBN "you can manage what requires auto- mation, make your system standardized and reliable, and ensure you’re free to move and adjust heading into the future." The promise of IBN is to do a change from node-to-node management to an autonomic system. "The sys tem self-operates, self-adjusts, and self-corrects within the parameters of your expressed technical objectives."
So everthing should work like this: you express your intent - this intent is translated and then orchestration configuration will roll out the changes onto your network.
I think on good phrase for IBN is: "You say what, it says how"

The second chapter is named "Looking at the characteristics of IBN. This chapter does not give any helpful information at all. One nice concept is mentionend here: "Simple Pane of Glass": "t’s an important concept and a valuable benefit of having a single source of truth: You can see your entire network from a single, consistent perspective." But is think this is not possible for networks. Depending on your perspective (ethernet, vlans, ips, mpls, ...) the view is completely different. Just think about hardware ports vs virtual ports...

"Detailing the IBN architecture" is the titel of chapter 3. This chapter is with 9 pages the biggest chapter inside the booklet. In this chapter an example is drilled through: The intent "I want a VLAN connecting servers A, B, C, and D." is analyzed and the steps from define, translate, verify, deploy and monitor are shown.
In addition there are some subsection where the reference design, abstractions, inventory are put into relation to each other. This is illustrated with very nice figures. Really a good chapter!

In chapter four the book moves forward from fulfillment to assurance. "This chapter shows you why your IBN system (IBNS) requires sophisticated, deep analytics that can detect when a deployed service is drifting out of spec and either automatically make the adjustments to bring it back into compliance or alert you to the problem."
It starts with differentiating uncontrolled changes from controlled changes. This is nothing special to IBN. I think this is useful for any kind of operation in IT.

Chapter 5 is as always in this "dummmies" series a recap of the chapters before.

All in all a nice booklet which introduces very well in this new kind of network management system. But if IBN can keep the promises - let's see...