Archives

bookmark_borderOctober Review: TOTP in Horde

Posted on by Ralf

I have been working on multiple things recently.

Kronolith Web UI: Appointment Cancellation Bug

Fix an annoying bug where internal user attendees get cancellation mails when an appointment is updated by the owner. This only seems to happen from the Web UI, not from CalDAV. I already analysed how this is happening. The fix is going to be a little bigger as I do not want to invest in the legacy infrastructure (socalled “Imples”) and use the opportunity to use a more modern approach. Work is in progress.

New Material UI based frontend for passwd.

I have worked with the team on a Material Design based UI. It uses ReactJs and Typescript and the new horde/http_server library and it is very different from existing Horde UIs. Do not expect it to blend well with the existing horde look&feel. The whole thing is a proof of concept and is an alien as the DIMP UI was back in Horde 3. This proof of concept still lives in a public feature branch and if you want to try it, you need to enable a new setting in the Preferences Screen.

Two-Factor support in Horde Base and a TOTP library

More and more online services start using two-factor authentication for improved security. Along with a password, users have to enter some passcode they read from a keychain fob device or from an app on their phones (like Google Authenticator).

I have started a new library horde/otp which implements TOTP and other styles of passcodes used as a secondary authentication factor. The library needs some additional glue code in horde/core and horde/base which still has to be built. I would have liked to finish this in October but there is only so much time.

Improved horde-installer-plugin

The composer plugin for Horde has received some refactoring and enhancements. The current feature branch offers a custom command in the composer CLI . This custom command rebuilds the relevant configuration files when you move your Horde installation after running the install/update commands. There are also some minor changes to the way configurations are written. End users should not notice.

DNS library

B1 Systems have finally opensourced a DNS library for the Horde ecosystem. It has been used internally for some years. The library can serve as the DNS building block of an IPAM system, but it also has an adapter to apply changes to the Amazon Route 53 service.

PHPStan support

Beginning this month, libraries and apps will gradually introduce the static analyzer tool phpstan. The tool will run as part of the CI pipeline and detect various types of code imperfections which potentially can mean hard-to-detect bugs. The findings will be addressed as time permits.

bookmark_borderDeveloper Introduction to Maintaina Horde

Posted on by Ralf

This introduction is targeted at developers with little or outdated prior Horde knowledge. Having worked with any Backend Development Framework in any language can help. While a lot of these facts can be found in Wikis or previous article, others are fairly new or only relevant to the developments in the version of horde delivered via https://horde-satis.maintain.com – This article will be continuously updated as I have time or relevant questions pop up.

Setup your work environment

A ready-made docker-compose deployment can be downloaded here: https://github.com/maintaina/deployments/
A basic root project for installing horde via composer can be found here: https://github.com/maintaina-com/horde-deployment/

Either way, you should be able to connect into a running sample installation on your local desktop within less than 5 minutes. Visual Studio Code and other editors can directly connect into the container content, so there is no need for fancy mounts etc. Mind each repo’s readme files for instructions.

Horde as viewed by the composer installer

Composer is the installer used for Horde and also does most of the autoloading.

A root project should have at the very least the paths /web/ for all web visible assets, /vendor/ for dependencies and /var/ for both configuration, logs and variable data which should NOT be web-visible.

Besides the default library type, horde comes with a composer plugin to support the types horde-application, horde-theme and horde-library. The /web/ and /vendor/ dirs are under the control of composer, do NOT put any custom content there. It will be deleted with each subsequent update. This is different from the older PEAR installer which would only overwrite files with newer package content, but not remove any custom files not included in the package.

A ready-made docker-compose deployment with some default configuration can be found here:

All horde-application packages are installed to the /web/ dir.
All library, horde-library, horde-theme packages will be installed to the /vendor/ dir.
If a package is a horde-library or a horde-application and has a js/ dir, its contents are symlinked into a structure below web/js/. If a package is a horde-application or a horde-theme, specific links under the /web/themes/ dir are created on install or update. The installer plugin will search for application config files under the /var/config path and link them to /web/$app/config/. The installer will also autogenerate some files under /var/config if they are missing.

Filesystem layout

All Horde applications share the same filesystem layout.

  • /app/controllers/ contains old-style Horde_Controller request handlers.
  • /bin contains commandline scripts or cron jobs, usually prepended by the application name and without the .php suffix – these will automatically linked to the /vendor/bin path unless otherwise declared.
  • /config contains actual defaults files from the package as well as symlinks to user-provided or autogenerated configuration items. The routes.php file goes here.
  • /doc dir contains the license file, changelog.yml file, optionally a CHANGES file and any documentation in RST format. Some documentation can be autogenerated from the horde.org wiki.
  • /js contains ready-to-run Javascript code. Minifying is not necessary as it can be done by horde.
  • /lib dir contains PHP code which is usually unnamespaced and follows the PSR-0 autoloading standard.
  • /locale contains machine-readable PO translation files and the sources from which they are generated.
  • /migration contains code for automated buildup, upgrade and teardown of SQL database schema.
  • /scripts contains upgrade scripts and non-php content like LDIF files or apache config snippets.
  • /src dir contains PHP code following PSR-4 Namespaced Autoloading and PSR-12 Coding Standards.
  • /templates contains PHP and HTML templates mostly for backend-rendered content
  • /test contains unit tests and integration tests built with the phpunit framework.
  • /themes contains css and image files. See separate section on themes.
  • The top dir contains a composer.json manifest, a PEAR package.xml manifest, possibly some README.md and other control files. Traditionally, it housed all the entry points through which the browser would call into PHP code and get server-rendered pages as a result. This is no longer the recommended pattern.

Most of your newly developed code should live somewhere under /src. Libraries follow the same layout.

Web Layout

The general rule is you cannot hardcode any web-visible paths. All assumptions will eventually be wrong.
In a default installation as generated by the installer, the horde base app will be under /horde/ and the login screen will be under /horde/login.php etc. Other applications will be on the same level besides the /horde base app – /turba for the addressbook, /passwd for the password utility etc. Javascript will be visible under /js and themes under /themes – this is, unless you have configured something else. The webroot could be /tools/foo/bar rather than /.
The webmail application could reside under https://imp.foo.org/ rather than https://foo.org/imp. Themes could be presented from a totally different subdomain. The horde backend knows how to autogenerate the appropriate URLs. Your frontend should not make any assumptions.

Translation

Use the horde-translation tool to maintain translations. It generates binary .po files from a human-readable format. PHP can use these files to translate English text to your chosen frontend language. You can expose your frontend language under language-independent keys filled with the appropriate translations. You should NOT maintain translations in your frontend code as translation files can be customized by the administrator.

Themes

Horde can apply themes at runtime. Theme CSS is applied in the following order:

  • Horde global default CSS
  • Horde global custom theme css
  • App-specific default CSS
  • App-specific custom theme css (if present)

This mechanism will break if you use a build process that prepends CSS definitions with build-specific prefixes.
User-provided themes have the composer type horde-theme. The installer will link them to the web/themes folder.

Caching and minifying Javascript

Horde has a runtime javascript bundler and minifier which will bundle all javascript marked for delivery into one file and minify it. This file will be cached into the web-visible static dir. On version update, the file name will change. For this to work, you need to tell the framework which javascript files to include rather than just dish out a static html file with hardcoded paths to the individual JS files.

TODO code example

Caching and minifying themes

Horde has a runtime css minifier and bundler. All CSS will be bundled into a file and minified. This file will be cached in the web-visible static dir. If the user selects another theme, horde will minify and cache a different collection of files. For this to work, you should not hardcode paths to CSS file paths in your HTML templates or frontend code.

TODO code example

The Registry

With this amount of configurability, you need a source of truth about what goes where. This is the Horde Registry.
The registry has an index of all applications, their base web and filesystem location, which RPC and Inter-App APIs they expose, how they are represented in the topbar menu and where their Javascript and Themes resources are available in the local filesystem and as viewed from the web.

Registry Configuration

Querying the Registry

Routes, Middlewares and Handlers

Each application may have a config/routes.php file which contains web-accessible routes relative to this application and.

use Horde\Core\Middleware\AuthHordeSession;
use Horde\Core\Middleware\RedirectToLogin;
use Horde\Passwd\Middleware\RenderReactApp;
use Horde\Core\Middleware\ReturnSessionToken;
use Horde\Core\Middleware\DemandAuthenticatedUser;
use Horde\Core\Middleware\DemandSessionToken;

use Horde\Passwd\Handler\ReactInit;
use Horde\Passwd\Handler\Api\ChangePassword;

$mapper->connect(
    'Api',
    '/api/changepw',
    [
        'controller' => ChangePassword::class,
        'stack' => [
            AuthHordeSession::class,
            DemandAuthenticatedUser::class,
            // DemandSessionToken::class,
        ],
    ]
);

$mapper->connect(
    'ReactInit',
    '/react',
    [
        'controller' => ReactInit::class,
        'stack' => [
            AuthHordeSession::class,
            RedirectToLogin::class,
        ]
    ]
);

This example from a development version of the passwd app shows two routes: /react would hand out a browser page with all the boilerplate to load a Single Page Application UI (SPA). /api/changepw would receive a JSON message and, if the request is valid, change the user’s password. Each route definition contains of a name, a URL pattern and a third parameter defining the controller, the middleware stack or constraints like only processing the route for POST request. A route can also contain placeholders that can expand into variables and defaults for optional parts. Routes are processed in a first-hit-wins strategy so place your specific route definitions before the general cases. All routes are interpreted as relative to the application’s webroot.

The controller can be any string representing a class which is either a traditional Horde_Controller, a PSR-15 Request Handler or a PSR-15 Middleware. The only requirement is that the Injector needs to know how to produce it, either via Autowiring or via some explicit Binder (Factory, Implementation, Annotation …).

The Stack is either an array of strings representing PSR-15 Middlewares or a string recognized by the injector which will produce an iterable list of Middlewares. If there is no stack parameter, a default stack will be applied: Only allow requests for authenticated users identified by session cookie, forward everybody else to the login page. If you do not want any middlewares before your controller, explicitly set stack to an empty array. The middleware stack will be executed top to bottom before the controller is called and the request can be modified in that phase. If any middleware decides to answer your request by itself, no further middlewares will be called and the controller will not be executed. Responses travel back upward through the middleware stack and can be modified during that phase.

Lifecycle of a browser session

A typical user session would go like this:

  • A user navigates to webroot or any application route
  • As his browser provides no valid session cookie, he is redirected to the login page
  • The user enters credentials into the login form and posts a request. This will create a valid session cookie
  • The user is forwarded to his desired route or a default route
  • The backend sends all the HTML, CSS, graphics and javascript required to show the requested page. Javascript and CSS are each minified and bundled into cache files. Also, the framework inserts a javascript variable with dynamic data like the location of the API endpoints, chosen UI language etc.
  • User interactions either trigger ajax requests to API routes or forward him to a location outside the Single Page Application (i.e. another horde app or someplace outside). For ajax requests which change backend content, another credential besides the cookie is needed, for example a custom header with the session key or a special write token.
  • Eventually, the user logs out and his session will be invalidated

Permissions system

Administrator users

Shares

Preferences

Configuration System

Virtual Host specific configuration

Inter-App API and RPC

Special classes

ORM Layer horde/rdo

bookmark_borderCustom iCalendar data in Kronolith & Nag

Posted on by Ralf

The iCalendar exchange format is everywhere in Horde’s calendar (kronolith) and tasks (nag) apps. It is offered for manual import/export. It is the centerpiece of the CalDAV synchronisation protocol and various APIs of these apps. The format also plays a role in email invitiations and updates sent via iTip. It is a very powerful and well-accepted format. This is, unfortunately, a little bit of a problem. Fortunately, we also have a solution

The iCalendar format was originally defined by the IETF. They released RFC 2445 back in 1998. While Horde still understands that format, a newer version 2.0 was defined in RFC 5545 in 2009. An extension to the 2.0 standard was released in 2016: RFC 7986 defines additional attributes needed to describe related conferencing software resources or other technical aspects.

Server and client software has always been free to add custom attributes to the different components of the standard. For example, Horde Kronolith stores and reads a special attribute X-HORDE-ATTENDEE to mark event attendees who have a horde user attached. Mozilla uses attributes like X-MOZ-LASTACK and MS Exchange comes with a long list of custom attributes, among them X-CALEND. Understanding these attributes when processing a icalendar file is optional. However, servers are expected not to silently drop attributes they do not understand.

Up until now, both Kronolith and Nag did support a wide range of attributes, but far from all. Events are reconstructed from calendar backend contents. Unsupported data simply gets lost. As it is technically impossible to know all attributes anybody out there may use, the opposite definition was needed: An attribute is an “other” attribute if it is not in the list of attributes the app already handles. Both apps got a catalog of attributes they understand and a property for storing everything else. Support for actually storing and retrieving this data is currently limited to the SQL backend. I do not currently invest time in Kolab support, maybe I will never do that.

Also, while I took care to ensure that normal editing via the UI will not break the “other” data, I did not cross check with scenarios where ActiveSync is involved. Please report bugs as you find them.

CalDAV event Storage

For kronolith, an optional storage of unmodified caldav events as received has been added during development. I am not sure if I will keep this or remove it again. It is useful for debugging various support scenarios but under normal operation, it is not required. It may be useful for creating a cache of external caldav calendars though.

Extend to Turba Addressbook

Turba Addressbook uses a similar data format vcard and implements the sister protocol CardDAV. Things are indeed a little more complicated. There are three relevant versions of vcard 2.1, 3.0 and 4.0 and multiple extension RFCs with additional contact attributes exist. Worse, there is a host of X- attributes from popular desktop addressbook vendors, Evolution, Kontact, Thunderbird, Android, MS Exchange/Outlook… There are at least two different ways to represent contact groups and the whole thing is a little messy. But the real problem lies with Turba’s highly configurable nature. At least two backends need to be upgraded: Both SQL storage and LDAP play a key role in typical Turba use cases. The exact layout of each addressbook can be different even inside the same installation. The list of vcard attributes considered as natively supported needs to be calculated for each addressbook and additional attributes maybe need to be stored outside of the specific backend. This will take some time and consideration.

Still, Turba’s tendency to forget what it does not care for needs to be addressed. Otherwise users risk losing attributes on sync.

bookmark_borderMaintaina Horde switches to openSUSE LEAP

Posted on by Ralf

Our Horde docker images have switched over from Tumbleweed to openSUSE LEAP once again.

Recently our container build CI job in github.com broke down unexpectedly. An investigation showed that Tumbleweed’s core libraries, especially libc, were too new for the CI’s build system, based on Ubuntu LTS.

This is the second time we abandoned the Tumbleweed basis for Horde docker containers. OpenSUSE Leap 15.3 uses a relatively old, but well-maintained, set of base libraries. Both Leap and Tumbleweed deliver PHP 7.4 as a basis for Horde. In both systems, we skip the packaged composer version for a static pick which we will update from time to time. We may switch over to packaged composer if we feel confident.

For users and administrators of the image, both Tumbleweed and Leap 15.3 should feel more or less the same. For end users of the delivered horde setup, there should not be any downsides. We will switch back to the Tumbleweed image in a while when we have picked a more recent version of Ubuntu.

bookmark_borderCardDAV vs MacOS Contacts

Posted on by Ralf

In case you run Horde, NextCloud or other CalDAV/CardDAV server products, sooner or later you will encounter users who want to use the MacOS Addressbook application, “Contacts”, to access their server contacts. As of MacOS 11.5.2, the apple contacts app only supports one addressbook per principal. It will pick the first addressbook and use it for reading and writing. Unfortunately, the first addressbook exposed by Horde’s carddav system often is the favourite recipients addressbook. This is readonly.

There is no real solution to this yet. You can try to trick your carddav server into having the “right” addressbook as the first in the list but this is about it.

If anybody has an idea how to tackle this limitation without breaking carddav for all the other clients, please let me know.

bookmark_borderCan Horde’s internal API use PSR-7 Messages?

Posted on by Ralf

The Horde Inter-App system has been around since Horde 3.

Horde Inter-App messages are addressed by a two part string. The first part, followed by a slash character, is called the API. The second part after the slash is called the method. Registry can delegate complete APIs to an application or a list of complete API/method strings. In the latter format, a certain API/method combination can be assigned to one app even if the API in general is assigned to another app. The API is implemented by a class $application_Api in file Api.php inside each horde application. That application class methods and their signatures are the methods exposed by the Inter-App API. There are some meta arrays controlling further details but let’s ignore them. All but a few APIs only take arrays and primitives (string, number, bool) as parameters and issue them as return types. This is because the RPC layer eventually receives and emits HTTP messages, which are just text. Only those Inter-App API methods which are meant to be strictly internal will consume and emit PHP Objects.

PSR-7 messages and PSR-15 handlers/middlewares are an interop standard. They do not make a lot of assumptions about the underlying implementation. They have been used to implement REST solutions as well as old style server-driven dynamic websites. The request objects contain an URI to a resource and will eventually result in a response object. In between there is usually a broker piece called a router, which analysis URI and other request parameter to assign it to the proper implementation code or chain of code pieces, called middlewares. Anywhere in that chain, an answer is created and sent back to the caller. The request and response bodies are streams of text or binary data, as the request and response are essentially text messages.

At first glance, this is an easy match. The Registry mediates between the message sent by the caller and the code which handles it. We could call it the router. The API class with its methods could be seen as a set of handlers. The PSR-7 ServerRequest object represents a HTTP request, but it also allows arbitrary attributes attached to the actual request data. These attributes may be any PHP value, including objects.

There are some details to keep in mind though.

The inter-app API has little definition on a data contract. It predates PHP method parameter types and return types. In traditional code, users could feed just about anything into the inter-app API and the implementation would need to guard against any value expected or unexpected. Inter-App API just assumes the caller is eligible to call. Authentication is delegated to the existing PHP session or to the RPC setup, authorization control must happen in the called code. That may lead to bloated, repetitive code in the implementation.

As each app has only one API class file, it is not currently possible to implement two different methods on different APIs if the same app handles it. If you have two different APIs clients/get and contracts/get and both are implemented in the same app, they will end up in the same code path. The way around it is with calls like clients/getClients and contracts/getContracts, but this is just ugly.

The rampage/routes/http_server stack can easily discern a GET /clients/ call and a GET /contracts call, but it only works inside a specific app. Setting up a separate API set of routes, we can easily have calls abstracted from the implementing app. The system of handlers and middlewares allows to delegate authentication and authorization checks outside of the actual implementation of an endpoint. This reduces repetitive boilerplate. One big issue remains. As of now, Inter-App can return native PHP objects to the caller. PSR-7 messages allow Attributes on the ServerRequestInterface but there is no equivalent in the ResponseInterface. Inter-App can carry objects (for internal calls) or serialisation-friendly nested arrays until it hits the RPC layer. This layer will turn it into a text structure, say XML or JSON. How would we do that in ResponseInterface implementations? How would that make the implementation reusable for a REST interface, app-internal AJAX or other code?

A vision of convergence

Bringing together new capabilities and existing system participants is tricky. A new RPC and Inter-App system should integrate with the old interface, it should not just stand beside it. Having two different Inter-App layers would be confusing, abandoning the old one right now would be unnecessary stress on developers’ time budgets.

As an inter-app user, I want to use $registry->call(‘method’, [params1, param2]) or $registry->api->method($param1, $param2…) as I did before.

As an RPC user, I want to call \Horde_Rpc::request('api/method', $params, $options) as I did before. I do not care what happens in the background.

As an application developer exposing an API, I do not want to give up Api.php right now, it has to work with the new stack as good or bad as it did before.

As a developer of new apps and features, I want to leverage extended capabilities. I want to be able to implement two distinct APIs using the same method names. I want to be able to return native objects, even serialisation-unfriendly ones with php resources, along with a serialisation-friendly message to use in RPC, Rest or other HTTP use cases. I do not want to be restricted to two levels of API/method. I want to re-use middleware I already built for the frontend AJAY.

As a distributed app developer, I want to define API resources and have them served either internally or by external microservices transparantly over http requests.

For the future, I would like some degree of introspection and possibly some guidance on allowed or required request parameters.

As an integrator, I want to securely communicate with only partially set up horde instances to finish or upgrade setups by firing HTTP requests.

Implementation approach

The horde-deployment project includes a route from webroot/api/ to a global API router managed by the horde base app. This API router is first populated by the Registry and then supplemented by a config/routes.api.php file in each registry app.

Regardless of the calling context, a cascade of middlewares sets attributes for the called API/route, the parameters, the resolved implementation and the outcome of already happened authentication checks. The implementation is either an adapter middleware calling an app’s Api class or an actual implementation middleware/stack. It will write the return values and other state into an attribute digested by the bottom of stack. In case of Inter-App, a token response is returned and the actual data structures are taken from the handler and returned to the caller. Real RPC backends generate appropriate headers and stream body for response. The response can possibly be processed further as it returns back to top of stack, for example gzip compressed or logged or trigger metrics updates.

bookmark_borderMove config out of webroot for Horde 6

Posted on by Ralf

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:


    

  • default registry.php file
    • 

  • default prefs.php file
    • 

  • routes.php file
    • 

  • conf.xml definition file
    • 

  • other defaults files like backends.php, nls.php
    • 

  • a stub for hooks (maybe we can get away without it)
    • 



These items reside outside the webroot:


    

  • conf.php file
    • 

  • registry.local.php
    • 

  • routes.local.php (if any, I have never seen it used)
    • 

  • horde.local.php
    • 

  • other overrides like prefs.local.php, backends.local.php
    • 

  • registry.d/ snippets
    • 

  • conf.d/ snippets
    • 

  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
    • 

  • Hooks Files. (We need to ensure correct autoloading though)
    • 







Support for existing installations






There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.






To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.


The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.




Outlook






Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.


Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:


    

  • default registry.php file
    • 

  • default prefs.php file
    • 

  • routes.php file
    • 

  • conf.xml definition file
    • 

  • other defaults files like backends.php, nls.php
    • 

  • a stub for hooks (maybe we can get away without it)
    • 



These items reside outside the webroot:


    

  • conf.php file
    • 

  • registry.local.php
    • 

  • routes.local.php (if any, I have never seen it used)
    • 

  • horde.local.php
    • 

  • other overrides like prefs.local.php, backends.local.php
    • 

  • registry.d/ snippets
    • 

  • conf.d/ snippets
    • 

  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
    • 

  • Hooks Files. (We need to ensure correct autoloading though)
    • 







Support for existing installations






There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.






To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.


The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.




Outlook






Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.


Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

  • Default assumptions in the Horde registry for where apps are located were wrong
  • Javascript from library packages needed to be web visible and Horde needed to know their path and webroot
  • Custom Themes location and webroot
  • Composer2 would wipe custom configuration during software updates
  • Composer2 would wipe the static dir during software updates

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

  • Point to /web/$app/ for an app’s fileroot
  • Fix the webroot assumption, too from $webroot/horde/$app to $webroot/$app
  • Make horde search for themes in /web/themes/$app/ rather than /web/horde/$themes and /web/horde/$app/$themes/
  • Make horde load horde and library javascript files from /web/js/horde rather than /web/horde/js

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation

These items remain in the config/ dir:

  • default registry.php file
  • default prefs.php file
  • routes.php file
  • conf.xml definition file
  • other defaults files like backends.php, nls.php
  • a stub for hooks (maybe we can get away without it)

These items reside outside the webroot:

  • conf.php file
  • registry.local.php
  • routes.local.php (if any, I have never seen it used)
  • horde.local.php
  • other overrides like prefs.local.php, backends.local.php
  • registry.d/ snippets
  • conf.d/ snippets
  • all vhost-specific overrides like conf-www.example.com.php, registry-www.example.com.php, prefs-www.example.com.
  • Hooks Files. (We need to ensure correct autoloading though)

Support for existing installations

There is currently no software level support for upgrading a Horde 5 installation from pear or git-tools to a Horde 6 installation. There is no Horde 6 release by Horde Upstream yet. Installations of the maintaina horde fork are treated as existing installations of Horde 6 for the purpose of this document.

To allow a smooth transition, horde-installer-plugin will be changed in this way:
For each file in the presets dir, check if it exists in the old and new location.
Normally, the old dir should not hold a copy of the file unless it is delivered by the package itself.
If the new /var/config/ folder is writeable, put the file there. Create subdirectories as necessary.
Finally, symlink all files from /var/config to corresponding old location, even if they are not from preset.

The logic for reading configs should prefer the new location, if available, but fall back to the old location. This must be addressed in some code paths in horde/core and possibly also somewhere in the individual apps.
Writing files should always go to the new location, though in theory the symlink should take care of that.

Outlook

Moving configuration out of the web-visible and composer-owned area is a major step forward. Future developments may reduce the web-visible surface of a horde installation even further.

Implementation progress

  • https://github.com/maintaina-com/horde-deployment as of 1.0.0 includes the var/config/ dir
  • https://github.com/maintaina-com/horde-installer-plugin as of 2.2.1 contains a working implementation. Configs from /presets/ are written to /var/config/ and symlinked to their traditional locations.

bookmark_borderHorde Authentication: Revisited

Posted on by Ralf

Horde Authentication as it is today has been devised more than a decade ago and it is surprisingly complex. Use cases and requirements have evolved. In a previous article, I introduced basic authentication and authorization needs and how they can be overwhelmingly complex to get right.

I will not go into details of how it currently works. Jan Schneider has done a very good series on this topic:

These articles written back in 2011 still reflect how things are done.

To sum it up:

  • There are different concepts of being logged in “to horde” and being logged in to a specific app
  • Horde uses Authentication Drivers for the initial login
  • Authentication Drivers offer different capabilities with impact on the ability to list or search users, create new users, change password etc.
  • Initial login can be either explicit through a redirect to login screen or implicit, called “transparent authentication”
  • While returning Horde sessions are transparent, resource backends like IMAP, LDAP, FTP etc may still need an explicit login per request.
  • Login sets up some orthogonal aspects of Horde like
    • presentation type (Dynamic, Traditional, Mobile, Minimal) – some restrictions and defaulting logic apply
    • Display Language and Localisation – also with some defaults of its own
    • Theme selection

Matters are complicated by Horde’s modular nature and a plethora of configuration options. Accessing the same installation through two different virtual hostnames may yield different authentication backends and different defaults or freedom in selecting themes.

Back when these design choices were made, nobody expected modern complexities to be relevant for a web application:

  • Normal users wanting to integrate desktop applications through CalDAV
  • Use of external trust providers and identity solutions like SAML, OpenID Connect
  • Prevalence of API use cases where other applications interact with Horde
  • Single Page Applications and Mobile Apps interacting with Horde like a remote system
  • Use of scope-reduced throwaway tokens to authenticate for limited purposes
  • Share token based guest access to specific resources

Let’s inspect some use cases.

I may want to authenticate my session by an external ID provider but still have some means of searching and listing other users when granting permissions to my resources.

I may want to rate-limit login attempts with bad passwords regardless of the authentication backend’s capabilities.
The same is true for enforcing a maximum passsword age.

When I make a REST API call, I want to explicitly avoid any state carried over from a previous request. Any session management is unwelcome overhead. Any logic dealing with presentation is just stealing response time.

Some APIs may accept either a Basic Authentication via username/password or some token ID.

Seeing the current number of unread mails on a portal page is welcome, but having to do an expensive imap login when I actually just look at the calendar is less desirable. More so, when the permission system does not even allow me to access the email component and the imap login is failing on each API call.

It seems important to isolate the different concerns from each other and break the mighty authentication logic into smaller, independent parts:

  • A User Repository for searching and listing known users
  • Actual validity check of a set of credentials, resulting in identification
  • User creation and deletion
  • Renaming existing users
  • Change Password to a chosen new password
  • Reset Password without choice

The implementation of these concerns should be strictly isolated from other aspects

  • Application Bootstrap: Setup of an environment, localization, specific view
  • normalization of a provided username, enforcing lowercase, adding a default domain or other
  • How the credential is delivered: The LDAP or SQL backend must not care if the password comes via HTML Form, HTTP Header or some CLI option.
  • Authorization related aspects like checking password age, bad login count, temporary or permanent blacklist of usernames, if the user is known as a super administrator
  • Access to backend resources.
  • Session expiration
  • Explicit logout

A UI login should only ever result in a login to an external technology like LDAP, IMAP, remote web resources etc if the current context actually uses these resources.

Recently I introduced the horde/http_server component which allows a middleware-based approach to these topics.
We can neatly move the different authorization filters into stacks of middlewares doing one thing at a time or adding a conditional middleware. We can reuse predefined stacks for common scenarios. We can delegate the composition of a plethora of options to some factory and configuration and keep our actual drivers very lean and our bootstrap procedure as concise as possible.

In one of the upcoming articles, I will provide some example cases.

bookmark_borderAuthentication & Authorization is complex

Posted on by Ralf

Could there be any more straight forward topic than authentication & authorization? The user provides user name and password and clicks “login”, the backend checks if credentials are valid. Invalid credentials are not authorized, valid credentials are authorized and identified (authenticated). End of story. Right? Well… in many cases, it’s not that trivial.

As a user, I want to be informed if I have to change my password soon.

As a security officer, I want accounts blocked for some time after a certain amount of failed login attempts. I also want passwords to expire after a certain time. I also want login sessions to expire if client IP address or browser identity changes.

As an integrator, I want to enhance the system to digest certificates, Shibboleth, SAML or OpenID Connect, Bearer Tokens, JWTs or even Kerberos Tickets.

As a support person, I want to silently normalize user login names, lowercase them or append domain names to login names.

As a usability consultant, I want to leverage the user database for UI, make user names searchable and browseable.

As a site administrator, I want to be able to filter out certain or most users from the backend even if they provide valid credentials or block login for all but a few users, i.e. for site maintenance.

As an innovator I want to join user bases from two different authentication sources and possibly migrate them on next login seamlessly, without them noticing.

As a sales person, I want to allow a limited guest user experience prior to login rather than force everbody to the login screen.

As a returning user, I want to transparently log in through a remember me cookie, but maybe make the application aware of that limited trust, asking for real login for sensitive operations.

As a developer, I want to be flexible and allow any combination of criteria. Users may login with a global password or a purpose-limited token, I want them to use a second factor like TOTP if they have set up one but pass if they haven’t, unless I don’t allow it.

For business reasons, I want to rate limit API access per hour, per day and per month with individual thresholds each.

As an auditor, I want each request’s authorization process to be logged for evidence.

There is a lot more to consider, but I will stop here.

Authentication is any means of making sure of a requester’s identity. The most common practice in computers asking for a username (identity) and a password (proof). Another common practice is asking some external authority we trust, an Identity Provider. When we look at a person’s passport to compare his photo or fingerprint with his actual face or finger, this involves the same aspect: The name written on the passport for identity. The photo or fingerprint for proof. And, implicitly, we trust the party who created the passport (Identity Provider) and maybe have some means to check the integrity of the document. But if somebody has no passport but a driver’s license, a club membership card, we might instead use this for verification.

Transparent Authentication is a special case where we can identify the user without explicitly interacting with him. This can be achieved whenever the request carries credible identifying information like a pre-established cookie, a certificate, a passport token whose integrity can be validated or other methods.

Authorization is any decision making if a requester has access to a resource. Simply being authenticated might not be enough. Guests without authentication may be eligible for certain areas of your application, but maybe not if they are from a certain IP range or country. A person may be too young or too old to use a certain facility. A person may be old enough to buy alcohol but cannot currently present a sufficient document to proof this. On the other hand, the bearer of a ticket may be eligible to visit some concert, with no interest in his actual identity. A software user may need to both be authenticated and part of a certain privilege group “administrators” to access a configuration screen.

Both requirements can be linked to each other, as well as all those aspects mentioned above.

In another article I will look at how Horde does it and discuss if this approach is still right for modern use cases.

bookmark_borderCardDAV: What is Turba’s true data model?

Posted on by Ralf

Turba Addressbook imports, exports and syncs to and from many formats. I discussed all the formats in a previous article.

You can even customize the addressbooks, make them contain extra fields or omit common fields. Saving and loading data from the addressbook backend is in some way one more such conversion. Turba can actually read and write to SQL and LDAP backends, to the Preferences System and to an outdated version of the Kolab Groupware Server and IMSP Servers. It can derive Addressbooks from Horde Groups, Favourite Recipients (usually from IMP) and stored search queries (virtual addressbooks).

This diversity of backends and configuration options comes at a cost. It is sometimes not clear what is the data model and who owns the data. This complicates synchronisation scenarios like CardDAV.

Let’s look at this in detail. LDAP backends might be considered the owner of data and Turba would be just a view to that data or a client with edit permissions. LDAP would be able to change data, add new contacts, delete contacts or change contact details without turba even knowing. This is less of a problem with the SQL driver as the SQL schema of Turba should never be consumed directly, rather through turba’s API or exchange formats. Groups are a primary culprit of not telling Turba any changes.

On the other hand, sync protocols may contain data the current turba configuration or backend does not recognize. CardDAV and its data format vCard may contain a wealth of properties and allow custom extensions fields. CardDAV servers are supposed not to simply drop/forget fields they do not recognize because that could make the client forget fields in the data it actually supports. On the other hand, a client may send an update of a contact missing a property which is stored in a previous version of the contact. We need to understand if the client simply cannot handle that property or if it is supposed to remove that property. Well-behaved clients should not strip properties they do not understand and well-behaved servers should not do that either.

Turba may support contact information which cannot be mapped to CardDAV/vCard but I am currently unaware of any. As vCard allows custom fields, at the moment I consider vCard the lingua franca.

Another tough scenario are references to other contacts or resources. CardDAV/vCard support multiple ways to indicate these and the backend may support a completely different model of referencing. How should we react on references to contacts which we do not (yet) have? Many of these problems also affect CalDAV sync in the kronolith calendar and nag tasks app.

I am ignoring ActiveSync and SyncML because I do no longer possess any syncml capable device and I have limited understanding of the ActiveSync code. It seems to have a static mapping between turba internal field names and ActiveSync field names.

To resolve all these issues, Turba needs to have its own data model and it needs to store synced data its backend is not interested in. It must not lose or forget any information when syncing back to the client. This is currently not the case.