Dietrich Schroff

Subscribe to Dietrich Schroff feed
Dietrich Schroffhttps://plus.google.com/101232877473830670008noreply@blogger.comBlogger441125
Updated: 15 hours 57 min ago

AWS: How to delete a static website via aws cli

Thu, 2018-11-22 14:22
After the creation of a static website in S3 via cli, now the deletion:

First try was:

$ aws s3api delete-bucket --bucket my.webtest



An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty
Ok. This will not work. First get the objects:

$ aws s3api list-objects --bucket my.webtest

{

    "Contents": [

        {

            "LastModified": "2018-11-17T19:18:53.000Z",

            "ETag": "\"e56b419be959169c15260cd721735e47\"",

            "StorageClass": "STANDARD",

            "Key": "index.html",

            "Owner": {

                "DisplayName": "d.schroff",

                "ID": "6c301aed95f62fb17532da6c93209c898a1e07051e520c6bb7fab30769cc495c"

            },

            "Size": 568

        }

    ]

}
and the bucket can be deleted:
$ aws s3api delete-bucket --bucket my.webtest
A crosscheck via web console:


And the website is not there anymore:

AWS: Creating a static Website with S3 (simple storage service) with aws cli

Sat, 2018-11-17 13:34
There is a nice tutorial how to create a static webpage with using Amazon S3:
https://docs.aws.amazon.com/AmazonS3/latest/dev/HostingWebsiteOnS3Setup.html

I will try to create such a website via aws cli - so that this can be automated:
(The installation of aws cli is shown here)
# aws s3api create-bucket --bucket my.webtest --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1

{

    "Location": "http://my.webtest.s3.amazonaws.com/"

}

Then create a website.json file:

$ cat website.json 

{

    "IndexDocument": {

         "Suffix": "index.html"

     },

     "ErrorDocument": {

          "Key": "error.html"

     }

 }

and run

$ aws s3api put-bucket-website --bucket my.webtest --website-configuration file://website.json

After that the web console should show:
and

Next step is to create the file policy.json:

$ cat policy.json 

{

   "Version":"2012-10-17",

   "Statement":[{

     "Sid":"PublicReadForGetBucketObjects",

         "Effect":"Allow",

       "Principal": "*",

       "Action":["s3:GetObject"],

       "Resource":["arn:aws:s3:::my-webtest/*"

       ]

     }

   ]

 }

and run

aws s3api put-bucket-policy --bucket my.webtest --policy file://policy.json

You can check via:
$ aws s3api get-bucket-policy --bucket my.webtest

{

    "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"PublicReadForGetBucketObjects\",\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::my.webtest/*\"}]}"

}
Via the web console:
 Then upload you html page:

$ aws s3 cp TestWebPage.html s3://my.webtest/index.html

upload: ./TestWebPage.html to s3://my.webtest/index.html  
 And here we go:


That was easy. Ok - a DNS resolution via Amazon route 53 is missing, but with these commands you are able to deploy a static website without clicking around...



Postings related to AWS:









AWS: Billing - how to delete a route 53

Thu, 2018-11-15 14:58
After playing around with AWS containers
i took a look at my billing page:

So let's delete this service.
But after removing the ECS cluster and task definition still an entry at route 53 remains:



The resource hostedzone/Z3JCO1N1BVHCKX can only be managed through servicediscovery.amazonaws.com (arn:aws:servicediscovery:eu-west-1:803404058350:namespace/ns-so7m3qbqbatzmlgn)


But the solution is the aws cli (for installation take a look here):
schroff@zerberus:~/AWS$ aws servicediscovery list-services
{

    "Services": [

        {

            "Id": "srv-46ffbkbwzupvblsb",

            "Arn": "arn:aws:servicediscovery:eu-west-1:803404058350:service/srv-46ffbkbwzupvblsb",

            "Name": "my-nginx-service"

        },

        {

            "Id": "srv-nicoewsbpufb3tlk",

            "Arn": "arn:aws:servicediscovery:eu-west-1:803404058350:service/srv-nicoewsbpufb3tlk",

            "Name": "my-ecs-service-on-fargate"

        }

    ]

}

schroff@zerberus:~/AWS$ aws servicediscovery delete-service --id srv-46ffbkbwzupvblsb
schroff@zerberus:~/AWS$ aws servicediscovery delete-service --id srv-nicoewsbpufb3tlk


and

schroff@zerberus:~/AWS$ aws servicediscovery list-namespaces

{

    "Namespaces": [

        {

            "Type": "DNS_PRIVATE",

            "Id": "ns-so7m3qbqbatzmlgn",

            "Arn": "arn:aws:servicediscovery:eu-west-1:803404058350:namespace/ns-so7m3qbqbatzmlgn",

            "Name": "local"

        }

    ]

}
Take the id and delete this namespace:
schroff@zerberus:~/AWS$ aws servicediscovery delete-namespace --id=ns-so7m3qbqbatzmlgn

{

    "OperationId": "4kdit33kf7kfuawscpfgifcrdktynen5-jog7l6h7"

}

And the the hosted zone was gone:

AWS: Installing aws cli (Amazon Web Service Commandline)

Wed, 2018-11-14 14:15
The management of aws can be done via the webpage (https://console.aws.amazon.com) or via aws cli.



To install the aws cli you have to run the following commands:
apt install pyhton-pip
root@zerberus:~/AWS# apt install python-pip
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut.
Statusinformationen werden eingelesen.... Fertig
Die folgenden Pakete wurden automatisch installiert und werden nicht mehr benötigt:
btrfs-tools geoip-database-extra libcryptui0a libjs-openlayers seahorse-daemon
Verwenden Sie »apt autoremove«, um sie zu entfernen.
Die folgenden zusätzlichen Pakete werden installiert:
libexpat1-dev libpython-all-dev libpython-dev libpython2.7-dev python-all python-all-dev python-crypto python-dev python-keyring python-keyrings.alt python-pip-whl python-secretstorage python-setuptools python-wheel
python-xdg python2.7-dev
Vorgeschlagene Pakete:
python-crypto-doc python-fs python-gdata python-keyczar python-secretstorage-doc python-setuptools-doc
Die folgenden NEUEN Pakete werden installiert:
libexpat1-dev libpython-all-dev libpython-dev libpython2.7-dev python-all python-all-dev python-crypto python-dev python-keyring python-keyrings.alt python-pip python-pip-whl python-secretstorage python-setuptools
python-wheel python-xdg python2.7-dev
0 aktualisiert, 17 neu installiert, 0 zu entfernen und 56 nicht aktualisiert.
Es müssen 31,2 MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 49,0 MB Plattenplatz zusätzlich benutzt.
Möchten Sie fortfahren? [J/n]
Holen:1 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 libexpat1-dev amd64 2.2.5-3 [122 kB]
Holen:2 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 libpython2.7-dev amd64 2.7.15~rc1-1 [28,2 MB]
Holen:3 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 libpython-dev amd64 2.7.15~rc1-1 [7.684 B]
Holen:4 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 libpython-all-dev amd64 2.7.15~rc1-1 [1.092 B]
Holen:5 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-all amd64 2.7.15~rc1-1 [1.076 B]
Holen:6 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python2.7-dev amd64 2.7.15~rc1-1 [286 kB]
Holen:7 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-dev amd64 2.7.15~rc1-1 [1.256 B]
Holen:8 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-all-dev amd64 2.7.15~rc1-1 [1.100 B]
Holen:9 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-crypto amd64 2.6.1-8ubuntu2 [244 kB]
Holen:10 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-secretstorage all 2.3.1-2 [11,8 kB]
Holen:11 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-keyring all 10.6.0-1 [30,6 kB]
Holen:12 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-keyrings.alt all 3.0-1 [16,7 kB]
Holen:13 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python-pip-whl all 9.0.1-2.3~ubuntu1 [1.652 kB]
Holen:14 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python-pip all 9.0.1-2.3~ubuntu1 [151 kB]
Holen:15 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-setuptools all 39.0.1-2 [329 kB]
Holen:16 http://de.archive.ubuntu.com/ubuntu bionic/universe amd64 python-wheel all 0.30.0-0.2 [36,4 kB]
Holen:17 http://de.archive.ubuntu.com/ubuntu bionic/universe amd64 python-xdg all 0.25-4ubuntu1 [31,3 kB]
Es wurden 31,2 MB in 7 s geholt (4.521 kB/s).
Vormals nicht ausgewähltes Paket libexpat1-dev:amd64 wird gewählt.
(Lese Datenbank ... 415946 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../00-libexpat1-dev_2.2.5-3_amd64.deb ...
Entpacken von libexpat1-dev:amd64 (2.2.5-3) ...
Vormals nicht ausgewähltes Paket libpython2.7-dev:amd64 wird gewählt.
Vorbereitung zum Entpacken von .../01-libpython2.7-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von libpython2.7-dev:amd64 (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket libpython-dev:amd64 wird gewählt.
Vorbereitung zum Entpacken von .../02-libpython-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von libpython-dev:amd64 (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket libpython-all-dev:amd64 wird gewählt.
Vorbereitung zum Entpacken von .../03-libpython-all-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von libpython-all-dev:amd64 (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python-all wird gewählt.
Vorbereitung zum Entpacken von .../04-python-all_2.7.15~rc1-1_amd64.deb ...
Entpacken von python-all (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python2.7-dev wird gewählt.
Vorbereitung zum Entpacken von .../05-python2.7-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von python2.7-dev (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python-dev wird gewählt.
Vorbereitung zum Entpacken von .../06-python-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von python-dev (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python-all-dev wird gewählt.
Vorbereitung zum Entpacken von .../07-python-all-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von python-all-dev (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python-crypto wird gewählt.
Vorbereitung zum Entpacken von .../08-python-crypto_2.6.1-8ubuntu2_amd64.deb ...
Entpacken von python-crypto (2.6.1-8ubuntu2) ...
Vormals nicht ausgewähltes Paket python-secretstorage wird gewählt.
Vorbereitung zum Entpacken von .../09-python-secretstorage_2.3.1-2_all.deb ...
Entpacken von python-secretstorage (2.3.1-2) ...
Vormals nicht ausgewähltes Paket python-keyring wird gewählt.
Vorbereitung zum Entpacken von .../10-python-keyring_10.6.0-1_all.deb ...
Entpacken von python-keyring (10.6.0-1) ...
Vormals nicht ausgewähltes Paket python-keyrings.alt wird gewählt.
Vorbereitung zum Entpacken von .../11-python-keyrings.alt_3.0-1_all.deb ...
Entpacken von python-keyrings.alt (3.0-1) ...
Vormals nicht ausgewähltes Paket python-pip-whl wird gewählt.
Vorbereitung zum Entpacken von .../12-python-pip-whl_9.0.1-2.3~ubuntu1_all.deb ...
Entpacken von python-pip-whl (9.0.1-2.3~ubuntu1) ...
Vormals nicht ausgewähltes Paket python-pip wird gewählt.
Vorbereitung zum Entpacken von .../13-python-pip_9.0.1-2.3~ubuntu1_all.deb ...
Entpacken von python-pip (9.0.1-2.3~ubuntu1) ...
Vormals nicht ausgewähltes Paket python-setuptools wird gewählt.
Vorbereitung zum Entpacken von .../14-python-setuptools_39.0.1-2_all.deb ...
Entpacken von python-setuptools (39.0.1-2) ...
Vormals nicht ausgewähltes Paket python-wheel wird gewählt.
Vorbereitung zum Entpacken von .../15-python-wheel_0.30.0-0.2_all.deb ...
Entpacken von python-wheel (0.30.0-0.2) ...
Vormals nicht ausgewähltes Paket python-xdg wird gewählt.
Vorbereitung zum Entpacken von .../16-python-xdg_0.25-4ubuntu1_all.deb ...
Entpacken von python-xdg (0.25-4ubuntu1) ...
python-secretstorage (2.3.1-2) wird eingerichtet ...
python-pip-whl (9.0.1-2.3~ubuntu1) wird eingerichtet ...
python-setuptools (39.0.1-2) wird eingerichtet ...
python-crypto (2.6.1-8ubuntu2) wird eingerichtet ...
python-keyring (10.6.0-1) wird eingerichtet ...
python-wheel (0.30.0-0.2) wird eingerichtet ...
python-keyrings.alt (3.0-1) wird eingerichtet ...
Trigger für doc-base (0.10.8) werden verarbeitet ...
1 hinzugefügte Doc-base-Datei wird verarbeitet...
Dokumente werden mit scrollkeeper registriert ...
libexpat1-dev:amd64 (2.2.5-3) wird eingerichtet ...
Trigger für man-db (2.8.3-2ubuntu0.1) werden verarbeitet ...
libpython2.7-dev:amd64 (2.7.15~rc1-1) wird eingerichtet ...
python-pip (9.0.1-2.3~ubuntu1) wird eingerichtet ...
python2.7-dev (2.7.15~rc1-1) wird eingerichtet ...
python-all (2.7.15~rc1-1) wird eingerichtet ...
python-xdg (0.25-4ubuntu1) wird eingerichtet ...
libpython-dev:amd64 (2.7.15~rc1-1) wird eingerichtet ...
python-dev (2.7.15~rc1-1) wird eingerichtet ...
libpython-all-dev:amd64 (2.7.15~rc1-1) wird eingerichtet ...
python-all-dev (2.7.15~rc1-1) wird eingerichtet ...
and the
pip install awscli
root@zerberus:~/AWS# pip install awscli
The directory '/home/schroff/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/schroff/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting awscli
  Downloading https://files.pythonhosted.org/packages/65/8c/9ac9abe65374653eb65de2fdaecc43e0b6940378e8fccec3a23fbfdc656b/awscli-1.16.53-py2.py3-none-any.whl (1.4MB)
    100% |████████████████████████████████| 1.4MB 798kB/s
Collecting docutils>=0.10 (from awscli)
  Downloading https://files.pythonhosted.org/packages/50/09/c53398e0005b11f7ffb27b7aa720c617aba53be4fb4f4f3f06b9b5c60f28/docutils-0.14-py2-none-any.whl (543kB)
    100% |████████████████████████████████| 552kB 1.6MB/s
Requirement already satisfied: PyYAML<=3.13,>=3.10 in /usr/lib/python2.7/dist-packages (from awscli)
Collecting rsa<=3.5.0,>=3.1.2 (from awscli)
  Downloading https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl (46kB)
    100% |████████████████████████████████| 51kB 4.0MB/s
Collecting colorama<=0.3.9,>=0.2.5 (from awscli)
  Downloading https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting s3transfer<0 .2.0="">=0.1.12 (from awscli)
  Downloading https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl (59kB)
    100% |████████████████████████████████| 61kB 3.6MB/s
Collecting botocore==1.12.43 (from awscli)
  Downloading https://files.pythonhosted.org/packages/c4/d8/242e75a2d9df95510883b65d95c26dab16d2980296c5437782e4f961fbb6/botocore-1.12.43-py2.py3-none-any.whl (4.8MB)
    100% |████████████████████████████████| 4.8MB 303kB/s
Collecting pyasn1>=0.1.3 (from rsa<=3.5.0,>=3.1.2->awscli)
  Downloading https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl (72kB)
    100% |████████████████████████████████| 81kB 5.0MB/s
Collecting futures<4 .0.0="">=2.2.0; python_version == "2.6" or python_version == "2.7" (from s3transfer<0 .2.0="">=0.1.12->awscli)
  Downloading https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Collecting jmespath<1 .0.0="">=0.7.1 (from botocore==1.12.43->awscli)
  Downloading https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl
Collecting urllib3<1 .25="">=1.20 (from botocore==1.12.43->awscli)
  Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)
    100% |████████████████████████████████| 122kB 4.7MB/s
Requirement already satisfied: python-dateutil<3 .0.0="">=2.1 in /usr/lib/python2.7/dist-packages (from botocore==1.12.43->awscli)
Installing collected packages: docutils, pyasn1, rsa, colorama, futures, jmespath, urllib3, botocore, s3transfer, awscli
Successfully installed awscli-1.16.53 botocore-1.12.43 colorama-0.3.9 docutils-0.14 futures-3.2.0 jmespath-0.9.3 pyasn1-0.4.4 rsa-3.4.2 s3transfer-0.1.13 urllib3-1.24.1
To use the command line you have to create access keys:



 

and now you can configure your aws cli:

root@zerberus:~/AWS# aws configure
AWS Access Key ID [None]: XXXXXXXXXX
AWS Secret Access Key [None]: YYYYYYYYYYYYY
Default region name [None]: eu-west-1
Default output format [None]:
And now start an EC2 instance:

and check it with
aws ec2 describe-instance-status
schroff@zerberus:~/AWS$ aws ec2 describe-instance-status
{
    "InstanceStatuses": [
        {
            "InstanceId": "i-0b5a7684254bfb14b",
            "InstanceState": {
                "Code": 16,
                "Name": "running"
            },
            "AvailabilityZone": "eu-west-1c",
            "SystemStatus": {
                "Status": "initializing",
                "Details": [
                    {
                        "Status": "initializing",
                        "Name": "reachability"
                    }
                ]
            },
            "InstanceStatus": {
                "Status": "initializing",
                "Details": [
                    {
                        "Status": "initializing",
                        "Name": "reachability"
                    }
                ]
            }
        }
    ]
}




AWS: Running a docker-image with ECS (part 3): Stop it!

Tue, 2018-11-13 14:53
After running a docker-image on ECS i tried to stop my service:



But after a view seconds the task was respawned:


Hmmm - "select cancel and update the service to stop the task"...
"Update the service" is not so difficult:
But there is no stop button.
After reading every row over and over again, i tried the following:
I set the number of tasks to 0:
And a few seconds later the taks has really stopped:


AWS: Running a docker-image with ECS (part 2)

Tue, 2018-11-13 14:06
After creating a task inside AWS ECS (see here) i got stuck in creating a cluster and running the task inside the cluster.

So i deleted the cluster and startet with this page:

And here we go:

I chose "nginx":





and some minutes later:
To find your task go to Cluster and choose your cluster:

 Open the tab "Tasks":

 and click on the Task name "6b...." or respectively your name:

Here you get the public ip, which you can use for a first contact with your task:



AWS: Running a docker-image with ECS

Mon, 2018-11-12 15:09
After reading some parts of the AWS documentation i decided to launch a docker-image via ECS - or better i will try to launch nginx.

Go to Amazon ECS and click on "Task Definitions":

 Then "Create new Task Definition"
 and then "FARGATE":


After adding a name you have to click "add container" and put in nginx + nginx:latest:

Then go back to  "Task Definitions" and choose "Actions"
 If you select "Run Task", you will end up with this window:


"Cluster: None Available" - so next step is to add a FARGATE cluster:






Running a task definition will be a task in another posting ;-)

AWS: Networking - Virtual Privat Cloud

Sun, 2018-11-11 14:06
After changing my AWS plans from docker to kubernetes, i decided to put the aws services inside a vpc (virtual private cloud).
With this decision my AWS services are not reachable from the internet - only my laptop can access them ;-)
Here the official pictures from aws:



Here is a list of customer gateway devices, for which amazon provides configuration settings:
  • Check Point Security Gateway running R77.10 (or later) software
  • Cisco ASA running Cisco ASA 8.2 (or later) software
  • Cisco IOS running Cisco IOS 12.4 (or later) software
  • Dell SonicWALL running SonicOS 5.9 (or later) software
  • Fortinet Fortigate 40+ Series running FortiOS 4.0 (or later) software
  • Juniper J-Series running JunOS 9.5 (or later) software
  • Juniper SRX running JunOS 11.0 (or later) software
  • Juniper SSG running ScreenOS 6.1, or 6.2 (or later) software
  • Juniper ISG running ScreenOS 6.1, or 6.2 (or later) software
  • Netgate pfSense running OS 2.2.5 (or later) software.
  • Palo Alto Networks PANOS 4.1.2 (or later) software
  • Yamaha RT107e, RTX1200, RTX1210, RTX1500, RTX3000 and SRT100 routers
  • Microsoft Windows Server 2008 R2 (or later) software
  • Microsoft Windows Server 2012 R2 (or later) software
  • Zyxel Zywall Series 4.20 (or later) software for statically routed VPN connections, or 4.30 (or later) software for dynamically routed VPN connections
The following requirements have to be met:
IKE Security Association (required to exchange keys used to establish the IPsec security association)
IPsec Security Association (handles the tunnel's encryption, authentication, and so on.)
Tunnel interface (receives traffic going to and from the tunnel) Optional
BGP peering (exchanges routes between the customer gateway and the virtual private gateway) for devices that use BGP
I do not own one of these devices, but i hope that the linux laptop can configured as customer gateway with appropriate ipsec settings.

So let's configure the VPC at AWS:


 And create a subnet for this vpc:



After that you have to add a virtual private gateway:




and attach it to your vpc:



You have to add a route from the VPC to your local network:


Then create a vpn connection:





 Then download the configuration:
and hurray: AWS provides a strongswan configuration:
After i downloaded the file an followed the instructions provided there, i was able to connect and the aws dashboard showed that the connection is up:


and on my local machine:
root@zerberus:~/AWS# ipsec status
Security Associations (1 up, 0 connecting):
     Tunnel1[1]: ESTABLISHED 3 seconds ago, 192.168.178.60[XX.YY.YY.XX8]...34.246.243.178[34.246.243.178]
     Tunnel1{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: cb84b8e5_i 488e669b_o
     Tunnel1{1}:   0.0.0.0/0 === 0.0.0.0/0

Docker and AWS: Is there really an AND? Moving to Kuberenetes

Sun, 2018-11-11 10:49
After my first steps into AWS i did not find a way to run docker-swarm at AWS without installing the software on my own. (take a look here). At least you have to add task definitions to your dockerfiles to let them run on ECS.
This is not really bad, but the idea was to move to a cloud provider and just run the microservices inside the cloud without caring about the infrastructure (Some people call this iaas or paas ;-) ).
But with ECS i am not convinced, that a cluster orchestrator like docker-swarm is included. Today everyone talks about kubernetes as cluster orchestrator. Last year as i read Kubernetes: Up & Running, there was a subchapter with:

But this has changed!
Amazon offer EKS:


So the next step is get a microservice with kubernetes on AWS working.
To do the docker setup once again only on servers which are running as EC2 compute nodes and not on my local virtualbox is not interesting. 

Last remark: EKS uses Kubernetes, which orchestrates Docker - so there is an AND for "Docker and AWS", AWS does not provide orchestration with docker-swarm, which was the orchestrator which i used.


AWS: Docker and AWS - creating my first EC2 compute node

Sat, 2018-11-10 22:11
My first idea after all my experiences with docker was to run my docker application with AWS.
After registration i searched for docker inside AWS and i only found this:
Hmm. Does not look like i expected. I thought, that i have just to upload my docker image and then i can select on which AWS compute nodes i want to run this image.

But let's give it a try.

First step is to install Docker on an Amazon Linux instance:

Startpoint is  https://console.aws.amazon.com/ec2/


At the left side in the top bar you have to choose the region, where you EC2 instance should be launched:

Then:

i choose the one with "the repositories include docker, PHP, MySQL, PostgreSQL, ..."








 You have to download the keypair. Otherwise you will not be able to connect to your machine!
And after a short time:


And here we go:
schroff@zerberus:~/AWS$ chmod 400 181111-first-aws-keypair.pem 

schroff@zerberus:~/AWS$ ssh -i 181111-first-aws-keypair.pem ec2-user@ec2-35-180-192-27.eu-west-3.compute.amazonaws.com



       __|  __|_  )

       _|  (     /   Amazon Linux AMI

      ___|\___|___|



https://aws.amazon.com/amazon-linux-ami/2018.03-release-notes/

14 package(s) needed for security, out of 30 available

Run "sudo yum update" to apply all updates.

[ec2-user@ip-172-31-47-127 ~]$


Amazon Web Services: A Start into AWS

Sat, 2018-11-10 06:23
After spending a lot of time with docker / docker swarm i decided to see, how this all works by using AWS.

First step is to do the registration (only registered users have access to the AWS documentation!):

The start is https://aws.amazon.com/








Oracle Database 18c: Installation via rpm (without runInstaller.sh)

Thu, 2018-11-01 05:25
Three months after writing the post Where to find the oracle-database-ee-18c-1.0-1.x86_64.rpm package? Oracle released the rpms on their download page:


https://www.oracle.com/technetwork/database/enterprise-edition/downloads/oracle18c-linux-180000-5022980.html

So let's give it a try.


First thing is to install the prerequisites:
# yum install oracle-database-preinstall-18c


...
=================================================================================================================================================
Package Arch Version Paketquelle Größe
=================================================================================================================================================
Installieren:
oracle-database-preinstall-18c x86_64 1.0-1.el7 ol7_latest 18 k
Als Abhängigkeiten installiert:
bc x86_64 1.06.95-13.el7 ol7_latest 114 k
bind-libs x86_64 32:9.9.4-61.el7 ol7_latest 1.0 M
bind-utils x86_64 32:9.9.4-61.el7 ol7_latest 204 k
compat-libcap1 x86_64 1.10-7.el7 ol7_latest 17 k
compat-libstdc++-33 x86_64 3.2.3-72.el7 ol7_latest 190 k
glibc-devel x86_64 2.17-222.el7 ol7_latest 1.1 M
glibc-headers x86_64 2.17-222.el7 ol7_latest 678 k
gssproxy x86_64 0.7.0-17.el7 ol7_latest 108 k
kernel-headers x86_64 3.10.0-862.9.1.el7 ol7_latest 7.1 M
keyutils x86_64 1.5.8-3.el7 ol7_latest 53 k
ksh x86_64 20120801-137.0.1.el7 ol7_latest 881 k
libICE x86_64 1.0.9-9.el7 ol7_latest 66 k
libSM x86_64 1.2.2-2.el7 ol7_latest 39 k
libX11 x86_64 1.6.5-1.el7 ol7_latest 606 k
libX11-common noarch 1.6.5-1.el7 ol7_latest 163 k
libXau x86_64 1.0.8-2.1.el7 ol7_latest 28 k
libXext x86_64 1.3.3-3.el7 ol7_latest 38 k
libXi x86_64 1.7.9-1.el7 ol7_latest 40 k
libXinerama x86_64 1.1.3-2.1.el7 ol7_latest 13 k
libXmu x86_64 1.1.2-2.el7 ol7_latest 70 k
libXrandr x86_64 1.5.1-2.el7 ol7_latest 27 k
libXrender x86_64 0.9.10-1.el7 ol7_latest 25 k
libXt x86_64 1.1.5-3.el7 ol7_latest 172 k
libXtst x86_64 1.2.3-1.el7 ol7_latest 20 k
libXv x86_64 1.0.11-1.el7 ol7_latest 18 k
libXxf86dga x86_64 1.1.4-2.1.el7 ol7_latest 18 k
libXxf86misc x86_64 1.0.3-7.1.el7 ol7_latest 19 k
libXxf86vm x86_64 1.1.4-1.el7 ol7_latest 17 k
libaio-devel x86_64 0.3.109-13.el7 ol7_latest 12 k
libbasicobjects x86_64 0.1.1-29.el7 ol7_latest 25 k
libcollection x86_64 0.7.0-29.el7 ol7_latest 40 k
libdmx x86_64 1.1.3-3.el7 ol7_latest 15 k
libevent x86_64 2.0.21-4.el7 ol7_latest 208 k
libini_config x86_64 1.3.1-29.el7 ol7_latest 62 k
libnfsidmap x86_64 0.25-19.el7 ol7_latest 49 k
libpath_utils x86_64 0.2.1-29.el7 ol7_latest 27 k
libref_array x86_64 0.1.5-29.el7 ol7_latest 26 k
libstdc++-devel x86_64 4.8.5-28.0.1.el7_5.1 ol7_latest 1.5 M
libtirpc x86_64 0.2.4-0.10.el7 ol7_latest 88 k
libverto-libevent x86_64 0.2.5-4.el7 ol7_latest 8.2 k
libxcb x86_64 1.12-1.el7 ol7_latest 210 k
lm_sensors-libs x86_64 3.4.0-4.20160601gitf9185e5.el7 ol7_latest 41 k
mailx x86_64 12.5-19.el7 ol7_latest 244 k
net-tools x86_64 2.0-0.22.20131004git.el7 ol7_latest 305 k
nfs-utils x86_64 1:1.3.0-0.54.0.1.el7 ol7_latest 407 k
psmisc x86_64 22.20-15.el7 ol7_latest 140 k
quota x86_64 1:4.01-17.el7 ol7_latest 178 k
quota-nls noarch 1:4.01-17.el7 ol7_latest 90 k
rpcbind x86_64 0.2.0-44.el7 ol7_latest 59 k
smartmontools x86_64 1:6.5-1.el7 ol7_latest 460 k
sysstat x86_64 10.1.5-13.el7 ol7_latest 310 k
tcp_wrappers x86_64 7.6-77.el7 ol7_latest 78 k
unzip x86_64 6.0-19.el7 ol7_latest 169 k
xorg-x11-utils x86_64 7.5-22.el7 ol7_latest 113 k
xorg-x11-xauth x86_64 1:1.0.9-1.el7 ol7_latest 29 k
Aktualisiert für Abhängigkeiten:
libstdc++ x86_64 4.8.5-28.0.1.el7_5.1 ol7_latest 303 k

Transaktionsübersicht
=================================================================================================================================================
Installieren 1 Paket (+55 Abhängige Pakete)
Aktualisieren ( 1 Abhängiges Paket)
...
These packages are installed on a minimal Oracle Linux system.

After that i started the installation of the rpm database package:
 [root@localhost oracle]# yum install -y oracle-database-ee-18c-1.0-1.x86_64.rpm 
Geladene Plugins: ulninfo
oracle-database-ee-18c-1.0-1.x86_64.rpm wird untersucht: oracle-database-ee-18c-1.0-1.x86_64
oracle-database-ee-18c-1.0-1.x86_64.rpm wird zum Installieren markiert
Abhängigkeiten werden aufgelöst
--> Transaktionsprüfung wird ausgeführt
---> Paket oracle-database-ee-18c.x86_64 0:1.0-1 markiert, um installiert zu werden
--> Abhängigkeitsauflösung beendet

Abhängigkeiten aufgelöst

=============================================================================================================
 Package                       Arch          Version       Paketquelle                                 Größe
=============================================================================================================
Installieren:
 oracle-database-ee-18c        x86_64        1.0-1         /oracle-database-ee-18c-1.0-1.x86_64        7.8 G

Transaktionsübersicht
=============================================================================================================
Installieren  1 Paket

Gesamtgröße: 7.8 G
Installationsgröße: 7.8 G
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installieren     : oracle-database-ee-18c-1.0-1.x86_64                                                 1/1
[INFO] Executing post installation scripts...
[INFO] Oracle home installed successfully and ready to be configured.
To configure a sample Oracle Database you can execute the following service configuration script as root: /etc/init.d/oracledb_ORCLCDB-18c configure
  Überprüfung läuft: oracle-database-ee-18c-1.0-1.x86_64                                                 1/1

Installiert:
  oracle-database-ee-18c.x86_64 0:1.0-1                                                                     

Komplett!

The documentation says about the configurtion:


and

 [root@localhost oracle]# /etc/init.d/oracledb_ORCLCDB-18c configure
Configuring Oracle Database ORCLCDB.
DB-Vorgang vorbereiten
8 % abgeschlossen
Datenbankdateien werden kopiert
31 % abgeschlossen
Oracle-Instanz wird erstellt und gestartet
32 % abgeschlossen
36 % abgeschlossen
40 % abgeschlossen
43 % abgeschlossen
46 % abgeschlossen
Erstellen von Datenbank wird abgeschlossen
51 % abgeschlossen
54 % abgeschlossen
Integrierbare Datenbanken werden erstellt
58 % abgeschlossen
77 % abgeschlossen
Aktionen nach Abschluss der Konfiguration werden ausgeführt
100 % abgeschlossen
Erstellen der Datenbank abgeschlossen. Einzelheiten finden Sie in den Logdateien in:
 /opt/oracle/cfgtoollogs/dbca/ORCLCDB.
Datenbankinformationen:
Globaler Datenbankname:ORCLCDB
System-ID (SID):ORCLCDB
Weitere Einzelheiten finden Sie in der Logdatei "/opt/oracle/cfgtoollogs/dbca/ORCLCDB/ORCLCDB.log".

Database configuration completed successfully. The passwords were auto generated, you must change them by connecting to the database using 'sqlplus / as sysdba' as the oracle user.
and a last check with sqlplus:
[oracle@localhost ~]$ sqlplus / as sysdba

SQL*Plus: Release 18.0.0.0.0 - Production on Thu Nov 1 11:21:50 2018
Version 18.3.0.0.0

Copyright (c) 1982, 2018, Oracle.  All rights reserved.


Verbunden mit:
Oracle Database 18c Enterprise Edition Release 18.0.0.0.0 - Production
Version 18.3.0.0.0

SQL> set lines 200
SQL> set pages 200
SQL> select * from v$instance;

INSTANCE_NUMBER INSTANCE_NAME     HOST_NAME                              VERSION        VERSION_LEGACY    VERSION_FULL    STARTUP_ STATUS       PAR    THREAD# ARCHIVE
--------------- ---------------- ---------------------------------------------------------------- ----------------- ----------------- ----------------- -------- ------------ --- ---------- -------
LOG_SWITCH_WAIT LOGINS       SHU DATABASE_STATUS     INSTANCE_ROLE        ACTIVE_ST BLO     CON_ID INSTANCE_MO EDITION FAMILY
--------------- ---------- --- ----------------- ------------------ --------- --- ---------- ----------- ------- --------------------------------------------------------------------------------
DATABASE_TYPE
---------------
          1 ORCLCDB      localhost.localdomain                          18.0.0.0.0    18.0.0.0.0          18.3.0.0.0    01.11.18 OPEN          NO       1 STOPPED
        ALLOWED    NO  ACTIVE         PRIMARY_INSTANCE   NORMAL    NO       0 REGULAR     EE
SINGLE

If you do not want to set a database with other options, you have to skip the
/etc/init.d/oracledb_ORCLCDB-18c configure
Just run the dbca and set up the options as you want.

jconsole/visualvm/java applications without fonts over ssh forwarding (characters displayed as boxes)

Sat, 2018-10-27 15:40
On servers which run java applications sometimes you need to run jconsole or jvisualvm.
This is typically no problem - ssh -X and you are done.

But in some rare circumstances you will get something like this:


??

Following some other sources, you end up with tweaking xorg parameter without any success. The solution is very easy:
apt-get install ttf-dejavuAnd then:


Java: Slow java with server.policy enabled - how to fix this issue

Sat, 2018-10-06 14:35
If you use Java security manager for hardening your java processes, you have to add the following JVM options:
-Djava.security.manager
-Djava.security.policy=server.policy Create a server.policy file (you can use jdkXXX/jre/lib/security/java.policy as a tamplate) and add the following line:
permission java.net.SocketPermission "localhost:*", "listen, accept, connect, resolve"; Now create a small java program, which listens on a port (like this example).

If you send a message with netcat
nc -u localhost 9876Everyhting is fine.
Now send a message from a remote host. This does not work - like expected.

Try it again with the following network tracing running (capturing all DNS packets):
tcpdump -i any port 53Cool. For each connect a DNS-Lookup is done.
This could be a problem for high performance systems or for systems, which have to running/reachable DNS-Servers. In the latter case all requests will be sent to localhost:53 and of course, localhost will not give any answer. (This is not true - there will be a "ICMP - not reachable", but no DNS answer.).
If you add now line with *.*, to allow the connection the server.policy file should contain the following lines:
permission java.net.SocketPermission "*:*", "listen, accept, connect, resolve";
permission java.net.SocketPermission "localhost:*", "listen, accept, connect, resolve"; 
Hmmm. The connection is allowed, but there still DNS requests happening. The problem is that "*:*" is behind the "localhost:*" because Java reads this file from bottom to top - so if you write it this way:
permission java.net.SocketPermission "localhost:*", "listen, accept, connect, resolve";
permission java.net.SocketPermission "*:*", "listen, accept, connect, resolve";
there are no DNS requests happening anymore.

If you still see DNS requests: Take a look at this file:
YourJDK/jre/lib/security/java.policy there are some entries with java.net.SocketPermission like:
permission java.net.SocketPermission "localhost:0", "listen"; Because java first checks this file, you have to remove such lines, to get rid of the DNS requests.

If you do not need to use DNS, you should remove dns in /etc/nsswitch.conf. But, then no domain lookup will succeed  on this machine anymore...

Linux: journalctl and systemd - better than /var/log/messages?

Fri, 2018-09-28 13:51
Nearly 8 years ago systemd was introduced on some Linux distribution (s. here). Last week i discovered some helpful commands, which i share with you.

If you want to take a look at kernel message (for example system boot), you command is
dmesg
The new equivalent is
journalctl -k
Ok - not really amazing.
But all of you know the message
See "systemctl status nginx.service" and "journalctl -xe" for details.
You can run the "systemctl start/restart/stop" and in case of error open the logs with "journalctl -xe". I would recommend to open a seperate shell and run there
journdalctl -f
This is something like "tail -f" to the systemd-journal.
If you do a "systemctl restart network" the shell with journalcctl -f shows the DHCP waiting for a answer from the server and you know why its so slow. You especially know, that your fifth interface has DHCP enabled and there is no DHCP, which slows down every "systemctl restart network".

journalctl has some nice filters like
journalctl -p 0..4This just shows the message with
  • "emerg" (0), 
  • "alert" (1), 
  • "crit" (2), 
  • "err" (3), 
  • "warning" (4), 
  • "notice" (5),
  • "info" (6), "debug" (7)
Or filter for something like network messages:
journalctl -u NetworkManager

And my favourite: Pipe your own log messages into the systemd-journal:
echo This is important | systemd-cat -t MightyJournal -p notice
Which result in this entry:
Sep 28 20:48:55 zerberus MightyJournal[28520]: This is important

Missing directory in /var/run or /run - tmpfiles.d

Wed, 2018-09-19 14:40
Sometimes is happens, that an application/demon refuses to start because of missing files/directories in /var/run.
The first solution is:
  • Create the directory in /var/run
  • Change the permissions
and everything is fine.

Not really.

After the next reboot, the directory is missing again and you have to go for the "first" solution again.

The right solution works like this:
Inside /usr/lib/tmpfiles.d create a myexample.conf file with this content:
        d /var/run/myexample 0755 schroff schroff -
To check if everything is ok run the following command:
        systemd-tmpfiles --create myexample.conf
and you will see:
# ls -l /var/run/ |grep mxexample
drwxr-xr-x  2 schroff schroff   40 19. Sep 22:45 myexample
And this directory will be created with each reboot...

MySQL 8: New Features

Sat, 2018-09-01 14:19
After installing mysql 8 on my ubuntu i did a lookaround for the new features. Oracle itself stated:

This is really cool, but how to test this?
A first look inside the standard data directory /var/lib/mysql shows:
/var/lib/mysql# ls -l
insgesamt 168012
-rw-r----- 1 mysql mysql       56 Aug 17 20:53 auto.cnf
-rw-r----- 1 mysql mysql      498 Aug 17 20:53 binlog.000001
-rw-r----- 1 mysql mysql      554 Aug 17 21:07 binlog.000002
-rw-r----- 1 mysql mysql       32 Aug 17 20:54 binlog.index
-rw------- 1 mysql mysql     1676 Aug 17 20:53 ca-key.pem
-rw-r--r-- 1 mysql mysql     1112 Aug 17 20:53 ca.pem
-rw-r--r-- 1 mysql mysql     1112 Aug 17 20:53 client-cert.pem
-rw------- 1 mysql mysql     1680 Aug 17 20:53 client-key.pem
-rw-r----- 1 mysql mysql     4533 Aug 17 20:53 ib_buffer_pool
-rw-r----- 1 mysql mysql 12582912 Aug 17 21:07 ibdata1
-rw-r----- 1 mysql mysql 50331648 Aug 17 21:07 ib_logfile0
-rw-r----- 1 mysql mysql 50331648 Aug 17 20:53 ib_logfile1
-rw-r----- 1 mysql mysql 12582912 Aug 17 20:56 ibtmp1
drwxr-x--- 2 mysql mysql     4096 Aug 17 21:07 mydatabase
drwxr-x--- 2 mysql mysql     4096 Aug 17 20:53 mysql
-rw-r----- 1 mysql mysql 25165824 Aug 17 21:07 mysql.ibd
drwxr-x--- 2 mysql mysql     4096 Aug 17 20:53 performance_schema
-rw------- 1 mysql mysql     1680 Aug 17 20:53 private_key.pem
-rw-r--r-- 1 mysql mysql      452 Aug 17 20:53 public_key.pem
-rw-r--r-- 1 mysql mysql     1112 Aug 17 20:53 server-cert.pem
-rw------- 1 mysql mysql     1680 Aug 17 20:53 server-key.pem
drwxr-x--- 2 mysql mysql     4096 Aug 17 20:53 sys
-rw-r----- 1 mysql mysql 10485760 Aug 17 21:07 undo_001
-rw-r----- 1 mysql mysql 10485760 Aug 17 21:07 undo_002
The sys and the perfomance_schema directories does not contain any frm or ISAM-style files. 
The mysql server team shows these pictures on their site (https://mysqlserverteam.com/mysql-8-0-data-dictionary-status-in-the-8-0-0-dmr/)



And if you create your own database each table is stored in its own ibd file and not all in one tablespace file, because innodb_file_per_table is set to 1 as default. With this shrinking tables can be done without a problem...






mysql 8: installation on ubuntu

Fri, 2018-08-31 14:01
Today i tried to install mysql 8 on my ubuntu:
First i visited the site https://dev.mysql.com/downloads/repo/apt/ which says:
So i downloaded this file (https://dev.mysql.com/get/mysql-apt-config_0.8.10-1_all.deb) and here we go:

# dpkg -i mysql-apt-config_0.8.10-1_all.deb 

Vormals nicht ausgewähltes Paket mysql-apt-config wird gewählt.

(Lese Datenbank ... 414911 Dateien und Verzeichnisse sind derzeit installiert.)

Vorbereitung zum Entpacken von mysql-apt-config_0.8.10-1_all.deb ...

Entpacken von mysql-apt-config (0.8.10-1) ...

mysql-apt-config (0.8.10-1) wird eingerichtet ...

Warning: apt-key should not be used in scripts (called from postinst maintainerscript of the package mysql-apt-config)

OK
with:
root@zerberus:~/Downloads# apt-get update

OK:1 http://de.archive.ubuntu.com/ubuntu bionic InRelease

Holen:2 http://de.archive.ubuntu.com/ubuntu bionic-updates InRelease [88,7 kB]

Holen:3 http://security.ubuntu.com/ubuntu bionic-security InRelease [83,2 kB]

Holen:4 http://repo.mysql.com/apt/ubuntu bionic InRelease [16,9 kB]

Holen:5 http://de.archive.ubuntu.com/ubuntu bionic-backports InRelease [74,6 kB]               

Holen:6 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [290 kB]

Holen:7 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 Sources [898 B]

Holen:8 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [147 kB]                    

Holen:9 http://de.archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [262 kB]                                            

Holen:10 http://de.archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [111 kB]                                                  

Holen:11 http://repo.mysql.com/apt/ubuntu bionic/mysql-apt-config amd64 Packages [568 B]                              

Holen:12 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 DEP-11 Metadata [138 kB]                               

Holen:13 http://de.archive.ubuntu.com/ubuntu bionic-updates/main DEP-11 48x48 Icons [31,4 kB]                           

Holen:14 http://de.archive.ubuntu.com/ubuntu bionic-updates/main DEP-11 64x64 Icons [53,7 kB]                        

Holen:15 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [161 kB]                         

Holen:16 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [119 kB]                           

Holen:17 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [161 kB]                      

Holen:18 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [57,4 kB]                                    

Holen:19 http://repo.mysql.com/apt/ubuntu bionic/mysql-apt-config i386 Packages [568 B]                                           

Holen:20 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [73,6 kB]                                     

Holen:21 http://security.ubuntu.com/ubuntu bionic-security/main amd64 DEP-11 Metadata [204 B]                                            

Holen:22 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [50,8 kB]  

Holen:23 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 DEP-11 Metadata [152 kB]                      

Holen:24 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [50,9 kB]                                 

Holen:25 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [29,4 kB]                                 

Holen:26 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 DEP-11 Metadata [5.792 B]                                

Holen:27 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe DEP-11 48x48 Icons [153 kB]                                    

Holen:28 http://security.ubuntu.com/ubuntu bionic-security/universe DEP-11 48x48 Icons [6.962 B]                                

Holen:29 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [1.608 B]                               

Holen:30 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [1.444 B]                           

Holen:31 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe DEP-11 64x64 Icons [262 kB]                   

Holen:32 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 i386 Packages [7.004 B]           

Holen:33 http://de.archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [3.772 B]      

Holen:34 http://de.archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [3.928 B]

Holen:35 http://de.archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 DEP-11 Metadata [2.468 B]

Holen:36 http://de.archive.ubuntu.com/ubuntu bionic-backports/universe amd64 DEP-11 Metadata [5.100 B]

Holen:37 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 amd64 Packages [7.002 B]          

Holen:38 http://repo.mysql.com/apt/ubuntu bionic/mysql-tools amd64 Packages [2.519 B]

Holen:39 http://repo.mysql.com/apt/ubuntu bionic/mysql-tools i386 Packages [1.882 B]

Es wurden 2.617 kB in 1 s geholt (2.002 kB/s).

Paketlisten werden gelesen... Fertig

root@zerberus:~/Downloads# apt-get install mysql-server

Paketlisten werden gelesen... Fertig

Abhängigkeitsbaum wird aufgebaut.      

Statusinformationen werden eingelesen.... Fertig

Die folgenden Pakete wurden automatisch installiert und werden nicht mehr benötigt:

  btrfs-tools esound-common gnome-dictionary gnome-icon-theme-symbolic libarmadillo7 libaudiofile1 libboost-date-time1.62.0 libboost-filesystem1.62.0

  libboost-iostreams1.62.0 libboost-random1.62.0 libboost-regex1.62.0 libboost-serialization1.62.0 libboost-system1.62.0 libboost-thread1.62.0

  libcapnp-0.5.3 libcaribou-gtk-module libcaribou-gtk3-module libdevhelp-3-4 libesd0 libfabric1 libfolks-telepathy25 libgeos-3.5.1 libgit2-24 libgl2ps1

  libgl2ps1.4 libgnome-games-support-1-2 libhdf5-openmpi-100 libhttp-parser2.1 libhwloc-plugins libhwloc5 libical2 libiso9660-8 libjsoncpp1

  libjsonrpc-glib-1.0-0 liblept5 liblivemedia58 libllvm5.0 liblouis12 liblouisutdml7 liblttng-ust-ctl2 libmirclient-dev libmircommon-dev libmircookie-dev

  libmircookie2 libmircore-dev libnetcdf-c++4 libnetcdf11 libntfs-3g872 libopencv-core3.1 libopencv-flann3.1 libopencv-imgproc3.1 libopencv-ml3.1

  libopencv-photo3.1 libopencv-shape3.1 libopencv-video3.1 libopencv-viz3.1 libopenmpi2 liborcus-0.12-0 libpoppler68 libprocess-cpp3 libprotobuf-dev

  libpsm-infinipath1 libqpdf18 librpm3 librpmbuild3 librpmio3 librpmsign3 libruby2.3 libsodium18 libsrtp0 libsuitesparseconfig4 libtbb2 libtesseract-data

  libtesseract3 libunity-api0 libva-drm1 libva-wayland1 libva-x11-1 libva1 libvlccore8 libvpx4 libvtk6.3 libx264-148 libx265-130 libxerces-c3.1

  libxkbcommon-dev libzmqpp4 openmpi-bin openmpi-common python-gi python3-libarchive-c python3-libnacl ruby2.3 snapd-login-service

Verwenden Sie »apt autoremove«, um sie zu entfernen.

Die folgenden zusätzlichen Pakete werden installiert:

  libmecab2 mecab-ipadic mecab-ipadic-utf8 mecab-utils mysql-client mysql-common mysql-community-client mysql-community-client-core mysql-community-server

  mysql-community-server-core

Die folgenden NEUEN Pakete werden installiert:

  libmecab2 mecab-ipadic mecab-ipadic-utf8 mecab-utils mysql-client mysql-community-client mysql-community-client-core mysql-community-server

  mysql-community-server-core mysql-server

Die folgenden Pakete werden aktualisiert (Upgrade):

  mysql-common

1 aktualisiert, 10 neu installiert, 0 zu entfernen und 36 nicht aktualisiert.

Es müssen 54,2 MB an Archiven heruntergeladen werden.

Nach dieser Operation werden 414 MB Plattenplatz zusätzlich benutzt.

Möchten Sie fortfahren? [J/n]

Holen:1 http://de.archive.ubuntu.com/ubuntu bionic/universe amd64 libmecab2 amd64 0.996-5 [257 kB]

Holen:2 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 amd64 mysql-common amd64 8.0.12-1ubuntu18.04 [78,6 kB]

Holen:3 http://de.archive.ubuntu.com/ubuntu bionic/universe amd64 mecab-utils amd64 0.996-5 [4.856 B]

Holen:4 http://de.archive.ubuntu.com/ubuntu bionic/universe amd64 mecab-ipadic all 2.7.0-20070801+main-1 [12,1 MB]

Holen:5 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 amd64 mysql-community-client-core amd64 8.0.12-1ubuntu18.04 [1.435 kB]

Holen:6 http://de.archive.ubuntu.com/ubuntu bionic/universe amd64 mecab-ipadic-utf8 all 2.7.0-20070801+main-1 [3.522 B]

Holen:7 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 amd64 mysql-community-client amd64 8.0.12-1ubuntu18.04 [2.292 kB]

Holen:8 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 amd64 mysql-client amd64 8.0.12-1ubuntu18.04 [75,7 kB]

Holen:9 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 amd64 mysql-community-server-core amd64 8.0.12-1ubuntu18.04 [16,9 MB]

Holen:10 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 amd64 mysql-community-server amd64 8.0.12-1ubuntu18.04 [21,0 MB]

Holen:11 http://repo.mysql.com/apt/ubuntu bionic/mysql-8.0 amd64 mysql-server amd64 8.0.12-1ubuntu18.04 [75,7 kB]                                          

Es wurden 54,2 MB in 10 s geholt (5.396 kB/s).                                                                                                             

Vorkonfiguration der Pakete ...

(Lese Datenbank ... 414916 Dateien und Verzeichnisse sind derzeit installiert.)

Vorbereitung zum Entpacken von .../00-mysql-common_8.0.12-1ubuntu18.04_amd64.deb ...

Entpacken von mysql-common (8.0.12-1ubuntu18.04) über (5.8+1.0.4) ...

Vormals nicht ausgewähltes Paket mysql-community-client-core wird gewählt.

Vorbereitung zum Entpacken von .../01-mysql-community-client-core_8.0.12-1ubuntu18.04_amd64.deb ...

Entpacken von mysql-community-client-core (8.0.12-1ubuntu18.04) ...

Vormals nicht ausgewähltes Paket mysql-community-client wird gewählt.

Vorbereitung zum Entpacken von .../02-mysql-community-client_8.0.12-1ubuntu18.04_amd64.deb ...

Entpacken von mysql-community-client (8.0.12-1ubuntu18.04) ...

Vormals nicht ausgewähltes Paket mysql-client wird gewählt.

Vorbereitung zum Entpacken von .../03-mysql-client_8.0.12-1ubuntu18.04_amd64.deb ...

Entpacken von mysql-client (8.0.12-1ubuntu18.04) ...

Vormals nicht ausgewähltes Paket libmecab2:amd64 wird gewählt.

Vorbereitung zum Entpacken von .../04-libmecab2_0.996-5_amd64.deb ...

Entpacken von libmecab2:amd64 (0.996-5) ...

Vormals nicht ausgewähltes Paket mysql-community-server-core wird gewählt.

Vorbereitung zum Entpacken von .../05-mysql-community-server-core_8.0.12-1ubuntu18.04_amd64.deb ...

Entpacken von mysql-community-server-core (8.0.12-1ubuntu18.04) ...

Vormals nicht ausgewähltes Paket mysql-community-server wird gewählt.

Vorbereitung zum Entpacken von .../06-mysql-community-server_8.0.12-1ubuntu18.04_amd64.deb ...

Entpacken von mysql-community-server (8.0.12-1ubuntu18.04) ...

Vormals nicht ausgewähltes Paket mecab-utils wird gewählt.

Vorbereitung zum Entpacken von .../07-mecab-utils_0.996-5_amd64.deb ...

Entpacken von mecab-utils (0.996-5) ...

Vormals nicht ausgewähltes Paket mecab-ipadic wird gewählt.

Vorbereitung zum Entpacken von .../08-mecab-ipadic_2.7.0-20070801+main-1_all.deb ...

Entpacken von mecab-ipadic (2.7.0-20070801+main-1) ...

Vormals nicht ausgewähltes Paket mecab-ipadic-utf8 wird gewählt.

Vorbereitung zum Entpacken von .../09-mecab-ipadic-utf8_2.7.0-20070801+main-1_all.deb ...

Entpacken von mecab-ipadic-utf8 (2.7.0-20070801+main-1) ...

Vormals nicht ausgewähltes Paket mysql-server wird gewählt.

Vorbereitung zum Entpacken von .../10-mysql-server_8.0.12-1ubuntu18.04_amd64.deb ...

Entpacken von mysql-server (8.0.12-1ubuntu18.04) ...

Trigger für ureadahead (0.100.0-20) werden verarbeitet ...

mysql-common (8.0.12-1ubuntu18.04) wird eingerichtet ...

Neue Version der Konfigurationsdatei /etc/mysql/conf.d/mysql.cnf wird installiert ...

Neue Version der Konfigurationsdatei /etc/mysql/my.cnf.fallback wird installiert ...

libmecab2:amd64 (0.996-5) wird eingerichtet ...

mysql-community-client-core (8.0.12-1ubuntu18.04) wird eingerichtet ...

mysql-community-server-core (8.0.12-1ubuntu18.04) wird eingerichtet ...

Trigger für libc-bin (2.27-3ubuntu1) werden verarbeitet ...

Trigger für systemd (237-3ubuntu10.3) werden verarbeitet ...

Trigger für man-db (2.8.3-2) werden verarbeitet ...

mecab-utils (0.996-5) wird eingerichtet ...

mysql-community-client (8.0.12-1ubuntu18.04) wird eingerichtet ...

mecab-ipadic (2.7.0-20070801+main-1) wird eingerichtet ...

Compiling IPA dictionary for Mecab.  This takes long time...

reading /usr/share/mecab/dic/ipadic/unk.def ... 40

emitting double-array: 100% |###########################################|

/usr/share/mecab/dic/ipadic/model.def is not found. skipped.

reading /usr/share/mecab/dic/ipadic/Noun.adverbal.csv ... 795

reading /usr/share/mecab/dic/ipadic/Adverb.csv ... 3032

reading /usr/share/mecab/dic/ipadic/Postp.csv ... 146

reading /usr/share/mecab/dic/ipadic/Suffix.csv ... 1393

reading /usr/share/mecab/dic/ipadic/Verb.csv ... 130750

reading /usr/share/mecab/dic/ipadic/Adnominal.csv ... 135

reading /usr/share/mecab/dic/ipadic/Prefix.csv ... 221

reading /usr/share/mecab/dic/ipadic/Noun.demonst.csv ... 120

reading /usr/share/mecab/dic/ipadic/Noun.csv ... 60477

reading /usr/share/mecab/dic/ipadic/Conjunction.csv ... 171

reading /usr/share/mecab/dic/ipadic/Noun.verbal.csv ... 12146

reading /usr/share/mecab/dic/ipadic/Noun.proper.csv ... 27327

reading /usr/share/mecab/dic/ipadic/Others.csv ... 2

reading /usr/share/mecab/dic/ipadic/Filler.csv ... 19

reading /usr/share/mecab/dic/ipadic/Noun.number.csv ... 42

reading /usr/share/mecab/dic/ipadic/Noun.org.csv ... 16668

reading /usr/share/mecab/dic/ipadic/Symbol.csv ... 208

reading /usr/share/mecab/dic/ipadic/Noun.nai.csv ... 42

reading /usr/share/mecab/dic/ipadic/Noun.adjv.csv ... 3328

reading /usr/share/mecab/dic/ipadic/Noun.place.csv ... 72999

reading /usr/share/mecab/dic/ipadic/Interjection.csv ... 252

reading /usr/share/mecab/dic/ipadic/Noun.name.csv ... 34202

reading /usr/share/mecab/dic/ipadic/Postp-col.csv ... 91

reading /usr/share/mecab/dic/ipadic/Noun.others.csv ... 151

reading /usr/share/mecab/dic/ipadic/Adj.csv ... 27210

reading /usr/share/mecab/dic/ipadic/Auxil.csv ... 199

emitting double-array: 100% |###########################################|

reading /usr/share/mecab/dic/ipadic/matrix.def ... 1316x1316

emitting matrix      : 100% |###########################################|



done!

update-alternatives: /var/lib/mecab/dic/ipadic wird verwendet, um /var/lib/mecab/dic/debian (mecab-dictionary) im automatischen Modus bereitzustellen

mysql-client (8.0.12-1ubuntu18.04) wird eingerichtet ...

mecab-ipadic-utf8 (2.7.0-20070801+main-1) wird eingerichtet ...

Compiling IPA dictionary for Mecab.  This takes long time...

reading /usr/share/mecab/dic/ipadic/unk.def ... 40

emitting double-array: 100% |###########################################|

/usr/share/mecab/dic/ipadic/model.def is not found. skipped.

reading /usr/share/mecab/dic/ipadic/Noun.adverbal.csv ... 795

reading /usr/share/mecab/dic/ipadic/Adverb.csv ... 3032

reading /usr/share/mecab/dic/ipadic/Postp.csv ... 146

reading /usr/share/mecab/dic/ipadic/Suffix.csv ... 1393

reading /usr/share/mecab/dic/ipadic/Verb.csv ... 130750

reading /usr/share/mecab/dic/ipadic/Adnominal.csv ... 135

reading /usr/share/mecab/dic/ipadic/Prefix.csv ... 221

reading /usr/share/mecab/dic/ipadic/Noun.demonst.csv ... 120

reading /usr/share/mecab/dic/ipadic/Noun.csv ... 60477

reading /usr/share/mecab/dic/ipadic/Conjunction.csv ... 171

reading /usr/share/mecab/dic/ipadic/Noun.verbal.csv ... 12146

reading /usr/share/mecab/dic/ipadic/Noun.proper.csv ... 27327

reading /usr/share/mecab/dic/ipadic/Others.csv ... 2

reading /usr/share/mecab/dic/ipadic/Filler.csv ... 19

reading /usr/share/mecab/dic/ipadic/Noun.number.csv ... 42

reading /usr/share/mecab/dic/ipadic/Noun.org.csv ... 16668

reading /usr/share/mecab/dic/ipadic/Symbol.csv ... 208

reading /usr/share/mecab/dic/ipadic/Noun.nai.csv ... 42

reading /usr/share/mecab/dic/ipadic/Noun.adjv.csv ... 3328

reading /usr/share/mecab/dic/ipadic/Noun.place.csv ... 72999

reading /usr/share/mecab/dic/ipadic/Interjection.csv ... 252

reading /usr/share/mecab/dic/ipadic/Noun.name.csv ... 34202

reading /usr/share/mecab/dic/ipadic/Postp-col.csv ... 91

reading /usr/share/mecab/dic/ipadic/Noun.others.csv ... 151

reading /usr/share/mecab/dic/ipadic/Adj.csv ... 27210

reading /usr/share/mecab/dic/ipadic/Auxil.csv ... 199

emitting double-array: 100% |###########################################|

reading /usr/share/mecab/dic/ipadic/matrix.def ... 1316x1316

emitting matrix      : 100% |###########################################|



done!

update-alternatives: /var/lib/mecab/dic/ipadic-utf8 wird verwendet, um /var/lib/mecab/dic/debian (mecab-dictionary) im automatischen Modus bereitzustellen

mysql-community-server (8.0.12-1ubuntu18.04) wird eingerichtet ...

update-alternatives: /etc/mysql/mysql.cnf wird verwendet, um /etc/mysql/my.cnf (my.cnf) im automatischen Modus bereitzustellen

Created symlink /etc/systemd/system/multi-user.target.wants/mysql.service → /lib/systemd/system/mysql.service.

mysql-server (8.0.12-1ubuntu18.04) wird eingerichtet ...

Trigger für systemd (237-3ubuntu10.3) werden verarbeitet ...

Trigger für ureadahead (0.100.0-20) werden verarbeitet ...
And everything is ready:
mysql -u root -pmysupersecretpassword

mysql: [Warning] Using a password on the command line interface can be insecure.

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 10

Server version: 8.0.12 MySQL Community Server - GPL



Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.



Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.



Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.



mysql>

systemd: systemd-notify not working for non-root-users

Sat, 2018-08-25 12:53
Sometimes you have to write your own startup scripts. Recent linux distributions require systemd scripts. This is not really a problem except you have to fulfill the following requirements:
  • Run the service as a non-root-user
  • The service has a startup phase and you want to start the next startup scripts after this startup phase
So the systemd-script has to look like this:
# cat /lib/systemd/system/TEST.service
[Unit]
Description=MyTestSystemdConfiguration

[Service]
User=schroff
Type=notify
ExecStart=/home/schroff/bin/test.sh
NotifyAccess=allThe service startup scripts have to look like this:
$ cat /home/schroff/bin/test.sh
#!/bin/bash

echo Starting serivce
sleep 10
#Starting your services
echo Services started

/bin/systemd-notify --ready
echo Notify done

while test 1 do
  sleep 600
done
#keep this scripts running, as long your service runsIn the startup phase you will get the following:
schroff@zerberus:~/bin$ systemctl status TEST.service
● TEST.service - MyTestSystemdConfiguration
   Loaded: loaded (/lib/systemd/system/TEST.service; static; vendor preset: enabled)
   Active: activating (start) since 19:39:27 CET; 7s ago
 Main PID: 17390 (test.sh)
    Tasks: 2 (limit: 4915)
   Memory: 532.0K::
      CPU: 7ms
   CGroup: /system.slice/TEST.service
           ├─17390 /bin/bash /home/schroff/bin/test.sh
           └─17395 sleep 10

19:39:27 zerberus systemd[1]: Starting MyTestSystemdConfiguration...
19:39:27 zerberus test.sh[17390]: Starting serivceAnd after the startup phase this is the output (if there were no errors):
schroff@zerberus:~/bin$ systemctl status TEST.service
● TEST.service - MyTestSystemdConfiguration
   Loaded: loaded (/lib/systemd/system/TEST.service; static; vendor preset: enabled)
   Active: active (running) since 19:38:38 CET; 3s ago
 Main PID: 17242 (test.sh)
    Tasks: 2 (limit: 4915)
   Memory: 932.0K
      CPU: 9ms
   CGroup: /system.slice/TEST.service
           ├─17242 /bin/bash /home/schroff/bin/test.sh
           └─17259 sleep 600

19:38:28 zerberus systemd[1]: Starting MyTestSystemdConfiguration...
19:38:28 zerberus test.sh[17242]: Starting serivce
19:38:38 zerberus test.sh[17242]: Services started
19:38:38 zerberus systemd[1]: Started MyTestSystemdConfiguration.
19:38:38 zerberus test.sh[17242]: Notify doneBut sometime you will get:
# systemctl restart TEST.service
Job for TEST.service failed because a timeout was exceeded.
See "systemctl  status TEST.service" and "journalctl  -xe" for details.19:44:46 zerberus systemd[1]: TEST.service: Start operation timed out. Terminating.
19:44:46 zerberus systemd[1]: Failed to start MyTestSystemdConfiguration.
-- Subject: Unit TEST.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit TEST.service has failed.
--
-- The result is failed.
19:44:46 zerberus systemd[1]: TEST.service: Unit entered failed state.
19:44:46 zerberus systemd[1]: TEST.service: Failed with result 'timeout'.
Note that this will happen after 600s (default). You can change this with the parameter (systemd configuration, see manpage systemd.service)
TimeoutSecBut changing this Parameter will not help, because systemd status will never enter the state "active (running)".

The problem is systemd-notify doesn't work, since it lives too short (Redhat Bugzilla).


A workaround is described in that bug entry:
Instead of
systemd-notify --readyuse
python -c "import systemd.daemon, time; systemd.daemon.notify('READY=1'); time.sleep(5)"

Pages