python3-threaded_servers

2023-05-10 21:51 UTC
  • Xyne

Metadata

Description: Threaded server modules (ThreadedHTTPSServer, ThreadedMulticastServer, Quickserve, Pacserve).
Latest Version: 2022.9
Source Code: src/
Architecture:
  • any
Dependencies:
  • python3
Optional Dependencies:
  • avahi: Avahi support
  • pyalpm: required for the Pacserve module
  • python-dbus: Avahi support
  • python-gobject: Avahi support
Arch Repositories:
  • [xyne-any]
  • [xyne-i686]
  • [xyne-x86_64]
AUR Page: python3-threaded_servers
Tags:

About

ThreadedServers is a collection of Python 3 modules for implementing various threaded servers.

Modules

ThreadedHTTPSServer.py
This modules subclasses the standard HTTP server and request handlers in http.server to provide threading support, HTTP Digest authorization, and HTTPS (i.e. SSL) support for both server and client certificates. The handlers also provide several convenience functions for transferring files (with multipart/byteranges support) and UTF-8 encoded content (HTML, JSON, plaintext).
ThreadedMulticastServer.py
A simple threaded subclass of socketserver.UDPServer for running a multicast server. The module includes a convenience function for sending multicast datagrams and a very basic example request handler.
Quickserve.py
The main backend file of quickserve.
Pacserve.py
The main backend file of pacserve.

Caveats

These modules are provided without any warranty whatsoever. The API may change unexpectedly from one release to another even if it tends to be stable. If you wish to use these modules in your own project, you should keep local copies of working versions to avoid migration issues.

Certificate Generation

Here are some references for creating self-signed certificates:

Generate a key and self-signed certificate for the server:

openssl req -newkey rsa:1024 -x509 -nodes -days 365 -keyout server.pem -out server.pem

Generate a certificate for the client using the server certificate created above (replace “My Certificate” with the name that you want to use):

openssl pkcs12 -export -out client.pfx -in server.pem -name "My Certificate"

This PKCS#12 certificate can then be used by clients to authenticate themselves when connecting to the server. For example, to import it in Firefox, “Edit” -> “Preferences” -> “Advanced” -> “View Certificate” -> “Your Certificates” -> “Import”.

Convert the PKCS#12 certificate to “pem” format:

openssl pkcs12 -in client.pfx -out client.pem -nodes

ThreadedServers.HTTPS Help Message

$ python3 -m ThreadedServers.HTTPS --help

usage: HTTPS.py [-h] [-a <interface|address>] [-p <port>]
                [--auth <string> <string>] [--authfile <filepath>] [--ssl]
                [--certfile <filepath>] [--keyfile <filepath>] [--req-cert]
                [--ca-certs <filepath>]

Run a basic HTTP(S) server with HTTP digest authentication support.

options:
  -h, --help            show this help message and exit

Server Address and Port:
  Configure the server's listening address and port.

  -a <interface|address>, --address <interface|address>
                        Bind the server to this address. By default the server
                        will listen on all interfaces.
  -p <port>, --port <port>
                        Set the server port (default: 8000)

HTTP Authentication:
  HTTP digest authentication via a username and password.

  --auth <string> <string>
                        HTTP digest username and password. Multiple pairs may
                        be passed.
  --authfile <filepath>
                        The path to a file containing alternating lines of
                        usernames and passwords.

SSL (HTTPS):
  Options for wrapping sockets in SSL for encrypted connections. Simply
  enabling SSL does not guarantee a secure connection and it is the user's
  responsibility to check that the implementation is correct and secure and
  that the server is properly configured. You can find information about
  generating self-signed certificates in the OpenSSL FAQ:
  http://www.openssl.org/support/faq.html

  --ssl                 Enable SSL (HTTPS).
  --certfile <filepath>
                        The path to the server's certificate.
  --keyfile <filepath>  The path to the server's key.
  --req-cert            Require a certificate from the client.
  --ca-certs <filepath>
                        Set the path to a file containing concatenated CA
                        certificates for verifying the client certificate.
                        This defaults to the server's own certificate.

ThreadedServers.Pacserve Help Message

$ python3 -m ThreadedServers.Pacserve --help

usage: Pacserve.py [-h] [--pacman-conf <filepath>] [--trust-pacserve-peers]
                   [--root <directory path>] [-f <filepath>]
                   [--filter <ix><regex>] [--filterlist <filepath>]
                   [--show-hidden]
                   [--tar {none,gz,bz2,xz} [{none,gz,bz2,xz} ...]]
                   [--upload <filepath>] [--allow-overwrite]
                   [--motd <filepath>] [--index <filename>]
                   [--peer <scheme>://<host>:<port>/] [--list-remote]
                   [--find-newest] [-a <interface|address>] [-p <port>]
                   [--auth <string> <string>] [--authfile <filepath>] [--ssl]
                   [--certfile <filepath>] [--keyfile <filepath>] [--req-cert]
                   [--ca-certs <filepath>] [--multicast]
                   [--multicast-server-address <interface|address>]
                   [--multicast-server-port <port>]
                   [--multicast-group <group>]
                   [--multicast-interval <seconds>]
                   [--multicast-interface <interface|address>]
                   [--multicast-ports <port> [<port> ...]] [--avahi]
                   [--avahi-interval <seconds>]
                   [<filepath> ...]

Pacserve.py - share Pacman packages over your LAN and beyond

positional arguments:
  <filepath>            Additional files and directories to share. These will
                        appear with the same name in server root. Use the
                        filelist option for more advanced features.

options:
  -h, --help            show this help message and exit

Pacserve Options:
  --pacman-conf <filepath>
                        The Pacman configuration file to use. Default:
                        /etc/pacman.conf
  --trust-pacserve-peers
                        Serve database and signature files and allow the
                        server to redirect database, signature and other non-
                        package request to its peers instead of immediately
                        redirecting to a mirror. This can be useful for some
                        setups but you should only use it if you trust the
                        peers or know exactly what you are doing.

File Download Options:
  --root <directory path>
                        If given then the directory will be treated as the
                        root of the server and all other paths will be
                        ignored. This is useful for testing static websites.
                        Similar and more complicated effects can be achieved
                        using a JSON filelist.
  -f <filepath>, --filelist <filepath>
                        A file to specify what to share on the server. If it
                        is a flat plaintext file then each line will be
                        treated as though it had been passed on the command
                        line. If it is a JSON file then it should be a map of
                        server paths to either single files or lists of
                        directories. The contents of each directory in the
                        list will appear as a single directory on the server.
  --filter <ix><regex>  Regular expressions to filter paths that appear on the
                        server. These will be applied in order when
                        determining which files to share.
  --filterlist <filepath>
                        A file consisting of filter expressions on each line.
                        The file will be reloaded if it is modified.
  --show-hidden         Share hidden files and directories.
  --tar {none,gz,bz2,xz} [{none,gz,bz2,xz} ...]
                        Enable directories to be transfered as optionally
                        compressed tar archives. This option accepts the
                        compression types to enable.

File Upload Options:
  --upload <filepath>   Enable uploads and save uploaded files in given
                        directory.
  --allow-overwrite     Allow uploaded files to overwrite existing files in
                        upload directory.

Content Options:
  --motd <filepath>     The MOTD message to display on the server. The file
                        will be reloaded if it is updated.
  --index <filename>    The name of the index page to display (if present)
                        when a directory is requested.

PeeredQuickserve Options:
  --peer <scheme>://<host>:<port>/
                        Static peers. Pass the option multiple times if
                        necessary. Example: "http://10.0.0.2:8000/"
  --list-remote         Include remote files in directory listings.
  --find-newest         Query all peers to find the newest version of a file
                        instead of returning the first one found.

Server Address and Port:
  Configure the server's listening address and port.

  -a <interface|address>, --address <interface|address>
                        Bind the server to this address. By default the server
                        will listen on all interfaces.
  -p <port>, --port <port>
                        Set the server port (default: 15678)

HTTP Authentication:
  HTTP digest authentication via a username and password.

  --auth <string> <string>
                        HTTP digest username and password. Multiple pairs may
                        be passed.
  --authfile <filepath>
                        The path to a file containing alternating lines of
                        usernames and passwords.

SSL (HTTPS):
  Options for wrapping sockets in SSL for encrypted connections. Simply
  enabling SSL does not guarantee a secure connection and it is the user's
  responsibility to check that the implementation is correct and secure and
  that the server is properly configured. You can find information about
  generating self-signed certificates in the OpenSSL FAQ:
  http://www.openssl.org/support/faq.html

  --ssl                 Enable SSL (HTTPS).
  --certfile <filepath>
                        The path to the server's certificate.
  --keyfile <filepath>  The path to the server's key.
  --req-cert            Require a certificate from the client.
  --ca-certs <filepath>
                        Set the path to a file containing concatenated CA
                        certificates for verifying the client certificate.
                        This defaults to the server's own certificate.

Multicast Options:
  Options that affect the behavior of the multicast (sub)server system.

  --multicast           Use multicasting to announce presence and detect other
                        servers.
  --multicast-server-address <interface|address>
                        The multicast server listening address. Default:
                        0.0.0.0.
  --multicast-server-port <port>
                        The multicast server listening port. Default: 15679.
  --multicast-group <group>
                        The multicast group. Default: 224.3.45.67.
  --multicast-interval <seconds>
                        The multicast announcement interval. Default: 300.
  --multicast-interface <interface|address>
                        The interface or address through which to announce
                        presence with multicast packets. If not given, all
                        interfaces on which the server is listening are used.
                        Interfaces on which the server is not listening are
                        ignored.
  --multicast-ports <port> [<port> ...]
                        The multicast ports to which to send announcement
                        messages. Default: 15679.

Avahi Options:
  Options that affect the behavior of the Avahi integration.

  --avahi               Use Avahi to announce presence and detect other
                        servers.
  --avahi-interval <seconds>
                        The avahi announcement interval. Default: 300.

CHANGELOG

2018-12-24

  • Server database and signature files when --trust-pacserve-peers option is enabled.
  • Add --find-newest option to all peered servers to query network for newest version.
  • Modified internal peer-to-peer protocol to pass around modification times to support --find-newest option.

2018-04-26

  • Modularized multicasting code and moved it from MulticastQuicksere.py to Multicast.py.
  • Changed multicast announcer code to ensure announcement is made on all interfaces if no interface is bound.
  • Renamed server address and port options for multicasting.
  • Added new option to configure multicasting announcement ports.
  • Renamed MulticastQuickserveServer to PeeredQuickserveServer
  • Renamed several functions and moved some to common.py
  • Added Avahi support.

2014-01-05

  • log file transfer sizes and local filepaths

2013-11-26

  • added new “multiple” attribute to file upload input tags
  • changed query strings for displaying custom pages (upload, peers)
  • added space before closing slash in empty XML tags
  • fixed bug in index page handling

2013-09-28

  • show all bound interfaces and addresses in info message

2013-07-01

  • support for multicasting via selected interfaces
  • accept interface names as binding address arguments

TODO

TODO

Figure out how to properly force all threads to exit.

  • finalize quickserve regex prefixes
  • add support for pseudo-directory to host autogenerated files (e.g. CSS)
  • create common base class for all servers
  • maybe cache generated pages locally (with MemoizeDB?)
  • add support for peerlist files
  • (re-?)add support for peer filters
  • optionally display peer files in local directory listings
  • client certificates for secure p2p networks
  • search function
  • peer exchange
  • logging options
  • per-directory authorizations

Page Generators

Create “get_page” methods to retrieve the page pre-str for other purposes, and move the str to the send_page method when necessary for consistency.

Contact
echo xyne.archlinux.org | sed 's/\./@/'
Validation
XHTML 1.0 Strict CSS level 3 Atom 1.0