Debian 8 Server mit Python über RESTful API erstellen

tutorial - debian Server mit python über RESTful API erstellen

Server mit python3 und RESTful API erstellen

Viele unserer Beispiele verwenden cURL dem einfachen Verständnis wegen. Aber deutlich mehr Kontrolle, eine bessere Integration in vorhandene Scripte und letztlich auch mehr Performance erhält man mit Sprachen wie zum Beispiel python.

Ich möchte mit diesem Guide keine komplette Library bauen, sondern pragmatisch die gridscale RESTful-API nutzen um einen Server mit python3 und der RESTful API zu erstellen.
Auch ist dieser Guide nicht als „Benchmark“ gedacht, sondern als Vorlage wie man die API mit python nutzen kann.
Daher führt die API-Befehle absichtlich sequentiell und mit großzügigen Poll-Intervallen aus um den Code verständlich und leicht reproduzierbar zu halten.

Was soll das Script alles können:
– eine passende Location für einen Server finden
– einen Server erstellen
– das Debian 8 Template finden
– ein Storage erstellen
– Storage und Public-Netzwerk mit dem Server verbinden
– den Server starten
– Server pingen und per SSH einloggen

Ok, los gehts 🙂

1) Script Grundgerüst
Damit User-ID und API-Token nicht im Script hardcoded werden, nutzen wir argparse und übergeben beide Werte beim Programmstart.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Create a server using the gridscale RESTful API.
"""
import argparse
import random
import string


def parse_args():
    """
    Setup parser for command line arguments.
    """
    apiurl = 'https://api.gridscale.io/objects'
    random_pw = ''.join(random.choice(string.ascii_lowercase + string.digits) for i in range(8))

    # create top level parser
    parser = argparse.ArgumentParser(description='Create a server using the gridscale RESTful API')

    parser.add_argument('userid',
                        type=str,
                        help='User-UUID')

    parser.add_argument('token',
                        type=str,
                        help='The API-Token (generate via https://my.gridscale.io/APIs/Create)')

    parser.add_argument('--apiurl',
                        type=str,
                        help='The API URL (default: ' + apiurl + ')',
                        default=apiurl)

    parser.add_argument('--rootpw',
                        type=str,
                        help='Server root password (default: random 8 char password)',
                        default=random_pw)

    return parser.parse_args()


def main(args):
    """
    Create a server with storage + public networking.

    1a) find a location we can use
     b) find the "Debian 8" template
     c) find the object_uuid of the public network
     d) request a new IPv4 and IPv6
    2) create a storage with the template
    3) create a server
    4) connect storage, public network, IPv4 and IPv6 to the server
    5) start the server
    """
    apiurl = args.apiurl
    headers = {'content-type': 'application/json'}
    headers['X-Auth-UserId'] = args.userid
    headers['X-Auth-Token'] = args.token


if __name__ == '__main__':
    main(parse_args())

2) RESTful API Status abfragen
Eine simple Funktion, deren einziger Zweck es ist, nachdem wir einen API-Request abgesetzt haben, die API auf Fertigstellung hin zu pollen.
Diese Funktion nutze ich um alles was wir mit der API machen strikt zu serialisieren. Natürlich erlaubt die API beliebige, parallele Requests – sofern diese nicht voneinander abhängen (Beispiel: ein Storage muss zuerst erstellt werden, bevor es mit einem Server verbunden werden kann). Threaded Verarbeitung ist etwas für eines der nächsten Guides.

import requests
import json
import time


def poll_state(apiurl, headers, url, urltype):
    """
    Basic RESTful API poller.

    Wait until a given endpoint reaches a certain "active" state.

    This helps us to serialize API requests.
    """
    poll_time_in_sec = 30
    sleep_time = 0.1
    counter = 0
    counter_max = int(poll_time_in_sec / sleep_time)

    while True:
        req = requests.get(apiurl + url, headers=headers)

        if req.status_code not in [200, 202, 204]:
            message = ['Polling failed - got status',
                       str(req.status_code),
                       'with error:',
                       str(req.text)]
            raise Exception(' '.join(message))

        data = json.loads(req.text)

        if data[urltype]['status'] in ['active']:
            break

        if counter > counter_max:
            message = ['Polling URL',
                       str(apiurl + url),
                       'timed out after',
                       str(poll_time_in_sec),
                       'seconds.']
            raise Exception(' '.join(message))
        time.sleep(sleep_time)

3) Location, Template und Public-Network finden

def choose_location(apiurl, headers, name):
    req = requests.get(apiurl, headers=headers)
    locations = json.loads(req.text)['locations']

    for location in locations:
        if locations[location]['name'] == name:
            return location

    raise Exception('Location not found.')

[...]
def main(args):
[...]
    # find the uuid of 'de/fra' location
    location_uuid = choose_location(apiurl, headers, 'de/fra')

def choose_template(apiurl, headers, name):
    req = requests.get(apiurl, headers=headers)
    templates = json.loads(req.text)['templates']

    for template in templates:
        if templates[template]['name'] == name:
            return template

    raise Exception('Template not found.')

[...]
def main(args):
[...]
    # find the uuid of the "Debian 8" template
    template_uuid = choose_template(apiurl, headers, 'Debian 8 (64bit)')

def choose_public_net(apiurl, headers):
    req = requests.get(apiurl, headers=headers)
    networks = json.loads(req.text)['networks']

    for network in networks:
        if networks[network]['public_net']:
            return network

    raise Exception('Public Network not found.')

[...]
def main(args):
[...]
    # find the uuid of the public network
    public_net_uuid = choose_public_net(apiurl, headers)

4) Eine IPv4 und eine IPv6 Adresse anfordern
IP-Adressen werden direkt vergeben, so dass kein Polling notwendig ist.
Auch kann es in einem normalen Szenario gut sein, dass man bereits IPs hat, welche man verwenden möchte.
Dann würde man hier natürlich nicht neue IPs anfordern, sondern mit einem GET auf /ips ganz ähnlich wie bei den locations eine Liste der vorhandenen IPs abfragen und sich davon die passende wählen.

def request_ip(apiurl, headers, ip_family, location_uuid):
    payload = {'location_uuid': location_uuid,
               'family': ip_family}

    req = requests.post(apiurl, data=json.dumps(payload), headers=headers)

    if req.status_code in [202]:
        return json.loads(req.text)

    raise Exception('IP could not be allocated')

[...]
def main(args):
[...]
    # request one IPv4 address
    ipv4_data = request_ip(apiurl, headers, 4, location_uuid)
    ipv4_uuid = ipv4_data['object_uuid']

    # request one IPv6 address
    ipv6_data = request_ip(apiurl, headers, 6, location_uuid)
    ipv6_uuid = ipv6_data['object_uuid']

5) Storage mit Debian 8 als Template erstellen

def create_storage(apiurl, headers, hostname, capacity, template_uuid, location_uuid, rootpw):
    payload = {'location_uuid': location_uuid,
               'name': 'Storage with Debian 8',
               'capacity': capacity,
               'template': {'hostname': hostname,
                            'template_uuid': template_uuid,
                            'password': rootpw,
                            'password_type': 'plain'}}

    req = requests.post(apiurl + '/storages', data=json.dumps(payload), headers=headers)

    if req.status_code not in [202]:
        raise Exception('Storage could not be created.')

    storage_data = json.loads(req.text)
    object_uuid = storage_data['object_uuid']

    poll_state(apiurl, headers, '/storages/' + str(object_uuid), 'storage')

    return object_uuid

[...]
def main(args):
[...]
    # create storage from Debian 8 template
    storage_uuid = create_storage(apiurl, headers,
                                  hostname='debian8'
                                  capacity=2,
                                  template_uuid=template_uuid,
                                  location_uuid=location_uuid,
                                  rootpw=args.rootpw)

6) Server konfigurieren

def create_server(apiurl, headers, name, cores, memory, location_uuid):
    payload = {'location_uuid': location_uuid,
               'name': name,
               'cores': cores,
               'memory': memory}

    req = requests.post(apiurl + '/servers', data=json.dumps(payload), headers=headers)

    if req.status_code not in [202]:
        raise Exception('Server could not be created.')

    storage_data = json.loads(req.text)
    object_uuid = storage_data['object_uuid']

    poll_state(apiurl, headers, '/servers/' + str(object_uuid), 'server')

    return object_uuid
[...]
def main(args):
[...]
    # create server
    server_uuid = create_server(apiurl, headers,
                                'Debian 8 Server',
                                cores=1,
                                memory=1,
                                location_uuid=location_uuid)

7) Alle Komponenten mit dem Server verbinden (Storage, Public-Netzwerk, IPv4, IPv6)

def add_relation(apiurl, headers, server_uuid, object_uuid, object_type):
    payload = {'object_uuid': object_uuid}

    if object_type == 'storage':
        reltype = '/storages'
    elif object_type == 'network':
        reltype = '/networks'
    elif object_type == 'ip':
        reltype = '/ips'

    request_url = apiurl + '/servers/' + str(server_uuid) + reltype
    req = requests.post(request_url, data=json.dumps(payload), headers=headers)

    if req.status_code not in [202]:
        raise Exception('Server could not be related to object.')

    poll_state(apiurl, headers, '/servers/' + str(server_uuid), 'server')

    return True

[...]
def main(args):
[...]
    # now all the server relations to
    #   - storage
    #   - public network
    #   - ipv4
    #   - ipv6
    add_relation(apiurl, headers, server_uuid, storage_uuid, 'storage')
    add_relation(apiurl, headers, server_uuid, public_net_uuid, 'network')
    add_relation(apiurl, headers, server_uuid, ipv4_uuid, 'ip')
    add_relation(apiurl, headers, server_uuid, ipv6_uuid, 'ip')

8) …und jetzt noch der letzte Schritt: Server anschalten

def power_on(apiurl, headers, server_uuid):
    payload = {'power': True}

    request_url = apiurl + '/servers/' + str(server_uuid) + '/power'
    req = requests.patch(request_url, data=json.dumps(payload), headers=headers)

    if req.status_code not in [202]:
        raise Exception('Server could not be started.')

    poll_state(apiurl, headers, '/servers/' + str(server_uuid), 'server')

    return True

[...]
def main(args):
[...]
    # finally power-on the server so we can use it
    power_on(apiurl, headers, server_uuid)

Fertig
Jetzt einmal das Script ausführen, die Zeit messen für Laufzeit des Scripts, warten bis der Server pingt und einen SSH-Login auf den neuen Server machen.
In den obigen Beispielen haben ich die Konsolen-Ausgaben weggelassen – im fertigen Script unten sind diese natürlich mit dabei.

Ausführen – los gehts 🙂

$ time ./create_server.py USER_ID TOKEN
Server root password: 9f3mnh8z
Choosing location...
Location found: 45ed677b-3702-4b36-be2a-a2eab9827950
Runtime was: 0.17123 seconds
---
Choosing template...
Template found: a5112c72-c9fa-4a23-8115-5f6303eb047b
Runtime was: 0.116985 seconds
---
Choosing public network...
Public network found: d20386b7-5892-447b-9e40-917d9b7a4c44
Runtime was: 0.102368 seconds
---
Requesting IPv4 ...
IPv4 found: 185.102.95.70
Runtime was: 0.203067 seconds
---
Requesting IPv6 ...
IPv6 found: 2a06:2380:0:1::26
Runtime was: 0.175562 seconds
---
Create storage...
Storage created: 7df35479-080d-4c04-9d07-3013742c4787
Runtime was: 6.554358 seconds
---
Create server...
Server created: 7cbfc4d6-4444-4222-be7d-182213c07aea
Runtime was: 0.288084 seconds
---
Server UUID: 7cbfc4d6-4444-4222-be7d-182213c07aea
Add storage to server 7cbfc4d6-4444-4222-be7d-182213c07aea
...done
Runtime was: 0.293474 seconds
---
Add network to server 7cbfc4d6-4444-4222-be7d-182213c07aea
...done
Runtime was: 0.27297 seconds
---
Add ip to server 7cbfc4d6-4444-4222-be7d-182213c07aea
...done
Runtime was: 0.314014 seconds
---
Add ip to server 7cbfc4d6-4444-4222-be7d-182213c07aea
...done
Runtime was: 0.334506 seconds
---
Power-ON Server 7cbfc4d6-4444-4222-be7d-182213c07aea
...done
Runtime was: 2.004724 seconds
---

real	0m10.997s
user	0m0.723s
sys	0m0.141s

$ ping 185.102.95.70
PING 185.102.95.70 (185.102.95.70) 56(84) bytes of data.
64 bytes from 185.102.95.70: icmp_seq=16 ttl=57 time=15.4 ms
64 bytes from 185.102.95.70: icmp_seq=17 ttl=57 time=14.7 ms
64 bytes from 185.102.95.70: icmp_seq=18 ttl=57 time=14.7 ms

$ ping6 -c 2 2a06:2380:0:1::26
PING 2a06:2380:0:1::26(2a06:2380:0:1::26) 56 data bytes
64 bytes from 2a06:2380:0:1::26: icmp_seq=1 ttl=57 time=16.0 ms
64 bytes from 2a06:2380:0:1::26: icmp_seq=2 ttl=57 time=15.8 ms

$ ssh root@185.102.95.70
Warning: Permanently added '185.102.95.70' (ECDSA) to the list of known hosts.
root@185.102.95.70's password: 

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
root@debian8:~# ping -c 2 www.google.de
PING www.google.de (173.194.113.15) 56(84) bytes of data.
64 bytes from fra02s19-in-f15.1e100.net (173.194.113.15): icmp_seq=1 ttl=60 time=0.912 ms
64 bytes from fra02s19-in-f15.1e100.net (173.194.113.15): icmp_seq=2 ttl=60 time=0.911 ms

--- www.google.de ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.911/0.911/0.912/0.030 ms
root@debian8:~#

Top – das ging ja einfach und schnell. Das komplette Script findet ihr hier: create_server.zip

Debian 8 Server mit python über RESTful API erstellen