horde – Page 5 – ralf-lang.de

Horde Config: How to fill dropdowns with application data with configspecial

Posted on February 22, 2012February 22, 2012 by Ralf

Horde provides system wide customisation and configuration of applications through php configuration files. These files can be edited by hand or written from an administrator config UI. This ui is automatically generated from a file called conf.xml located in your $application/config/ directory.

The config xml allows dropdowns, multiselect fields, tick boxes, radio buttons and even conditionally adding or removing a field or inserting a valid php expression.

For example a dropdown box in the horde base application’s config is generated by this snippet:

<configenum name="use_ssl" quote="false" desc="Determines how we generate
  full URLs (for location headers and such).">2
   <values>
    <value desc="Assume that we are not using SSL and never generate https
    URLs.">0</value>
    <value desc="Assume that we are using SSL and always generate https
    URLs.">1</value>
    <value desc="Attempt to auto-detect, and generate URLs
    appropriately">2</value>
    <value desc="Assume that we are not using SSL and generate https URLs only
    for login.">3</value>
   </values>
  </configenum>

This is all nice but what if you need to provide application data rather than static values? The answer is configspecial

<values>
    <configspecial application="turba" name="sources" />
</values>

How does that work?

<configspecial> calls the horde api. the “application” part tells you which application’s api to call. You can either reference an application by its registry name (horde, imp, kronolith, turba…) or by its api name (horde,mail, calendar, addressbook)

What’s the difference? When you call turba, you get turba. When you call addressbook, you can hook into whatever application provides addressbook. For example, spam handling and ticket queues have been implemented by multiple applications. You can even implement your own handlers for any existing api.

The called application must have a method configSpecialValues() in its lib/Application.php class file. This method gets called and its only parameter is the “name” property from the xml. In our example it’s “sources”. This method will return an array of source names to use in your config screen.

    /**
     * Returns values for <configspecial> configuration settings.
     *
     * @param string $what  The configuration setting to return.
     *
     * @return array  The values for the requested configuration setting.
     */

    public function configSpecialValues($what)
    {
        switch ($what) {
        case sources:
            try {
                $addressbooks = Turba::getAddressBooks(Horde_Perms::READ);
            } catch (Horde_Exception $e) {
                return array();
            }
            foreach ($addressbooks as &$addressbook) {
                $addressbook = $addressbook['title'];
            }

            $addressbooks[''] = _("None");
            return $addressbooks;
        }
    }

Et voila – you have a list of addressbooks to choose from.

Distributed applications with Horde 4

Posted on February 18, 2012December 6, 2012 by Ralf

Synopsis

Horde’s powerful RPC API has been used numerous times to allow integration of horde-based data into external applications or remote sites. It also provides an easy to set up basis for distributed applications with headless workers. In this article I will give you a brief introduction on how to build a scalable distributed architecture based on Horde 4.

Distributed Architecture

Assumptions:

You want your application to be scalable over several hosts. We call the controlling instance the master and the reacting instances the workers.
You don’t want to keep a lot of state on the worker. Adding or removing a worker instance should not require complicated setup. Most cloud layers like OpenStack assume worker instances to be virtually stateless. The master is the single source of truth and should be able to rebuild any broken or lost worker setup from stored information.
You are working in a hostile environment, e.g. the internet. Firewall only allows select ports and data has to travel over lines you cannot trust. You want to resort to https transport with real certificates.

The master:

I won’t go into too many details on the master setup this time. Create a basic app from the skeleton as the horde wiki describes. Separate a communication driver for worker Api calls from the driving logic in your app and don’t couple them too tightly. Usually you want small commits of changes to both the master’s idea and the worker’s reality and you want to check back if everything worked out. This doesn’t scale well on large-scale changes though.

Sometimes you want to make complex changes to the “truth” or “theory” in the master’s db before you commit them to the worker world out there.

Accessing the worker from the master:

The core piece of your communication with the worker are just a few lines of code

   protected function callWorker(WorkerInstance $worker, $callMethod, array $parameters = array()) {

       try {
            $http = new Horde_Http_Client(array('request.username' => $worker->rpcuser, 'request.password' => $worker->rpcuserpass, 'request.timeout' => 20 ));
            $response = Horde_Rpc::request(
                    'xmlrpc',
                    'https://' . $worker->worker_hostname . '/' . $worker->worker_subdir .'/rpc.php',
                    $callMethod,
                    $http,
                    array($parameters)
            );
        }
        catch (Exception $e) {
            throw new Appname_Exception($e);
        }
        return $response;
    }

This is a dumbed down version for demonstration purposes. You might want to model WorkerInstance based on Horde_Rdo, the horde ORM layer. It is desirable to evaluate lazy relations and lazy attributes. This has important performance implications but more on this in another post. We’re also selling consulting 😉

Worker setup:

We want a stateless worker instance. Obviously, this is theory. Truth is: You need a unique IP and you probably want a unique hostname. Nowadays cloud layers can provide that level of configuration. How about a horde instance without db?

horde/config/registry.local.php

You want the worker to talk under a specific api name. Add a block to your registry.local.php

 'myvpnworkerworker' => array (
        'name' => _("someworkerfooname"), /* we can even drop the _() as nobody will localize this */
        'provides' => 'myvpnworkerapp',
    )

horde/config/conf.php

This is stripped down to just the important lines

$conf['auth']['params']['htpasswd_file'] = '/not/in/webroot/passwords.secret';
 $conf['auth']['params']['encryption'] = 'plain'; /* In real world, you want to use some encryption instead */
 $conf['auth']['driver'] = 'http'; /* We want authentication by http layer after all */

We want the server to be stateless and not to rely on external data. We don’t want a local mysqld running and we don’t want a remote ldap either. We will store the credentials in a .htpasswd style file. For demonstration purposes, we use plain authentication.

The file would look like this:

passwd.passwd would look like this: 

rpcuser:totallysecretrpcuserpass
adminuser:adminpass
localdebuguser:secretlocaldebugpass

We also want to get rid of any components which cannot work without an sql backend

$conf['log']['priority'] = 'DEBUG';
$conf['log']['ident'] = 'HORDE';
$conf['log']['name'] = LOG_USER;
$conf['log']['type'] = 'syslog';
$conf['log']['enabled'] = true;
$conf['log_accesskeys'] = false;

As the worker will probably only show the admin UI to localhost or VPN, you want to log any debug relevant data locally into a file

$conf['prefs']['driver'] = 'Session';
$conf['alarms']['driver'] = false;

We don’t want user prefs or alarms on the worker. You might consider setting up some basic email delivery and sending alarms by mail. I won’t cover this here.

$conf['datatree']['driver'] = 'null';
$conf['group']['driver'] = 'Mock';

Datatree support is sql-only. Datatree is mostly legacy support and it isn't particularly fast either. There is no guarantee future horde revisions will support datatree. You don't want it. Period. You don't want groups either. The primary user of your instance is the RPC user.

$conf['perms']['driver'] = 'Null'

Only the master speaks to your worker and this must be ensured on the ssl/https layer. No need for a perms backend

$conf['cache']['driver'] = 'File';

If we use caching at all, we want to use a primitive one.

$conf['lock']['driver'] = 'Null';
$conf['token']['driver'] = 'Null';

Horde_Locks is a cool library. Ben Klang wrote it in 2008 when I was working in a non-public project that needed it and I mailed some stuff to him. But it’s sql-only. We don’t want it here.
Horde_Tokens are essential for a lot of verification tasks but the worker is not the single source of truth.

$conf['vfs']['type'] = 'File';

You probably don’t want a vfs at all. Vfs means state.

$conf['sessionhandler']['type'] = 'Builtin';

Anything but sql. You probably don’t want sessions.

This should be the key parts to make your stock horde installation not want a database at all.

The RPC Worker app.

The key to your RPC worker app is Api.php

This is the entry point for any Horde RPC calls.

Basically it works this way:

The upper layer of array() is internal to the horde rpc request layer
In our client example we wrapped our params into an additional array() to facilitate optional parameters. This means any method in Api.php accepts an array as the single parameter. You could also use a fixed list of parameters with optionals in the rear positions.
While the horde registry calls applications apis as ‘domain/function’, the rpc api calls them as domain.function. Examples are horde.listApis and myvpnapp.fetchData

Any function you can call from the outside is a method in Fooworkername_Api in Fooworkername/lib/Api.php.

Concurrency and queueing:

Horde is written in PHP. PHP is generally lacking in thread safety and doesn’t support real forking from within an apache module. You can however fork and detach processes using shell_exec. Horde ships some classes which help you use PHP in a shell environment but sometimes you want to resort to shell scripts or perl or anything else because it already exists or is more suitable to the job. shell_exec allows you use all of these. Usually you want your api calls to return fast. This doesn’t scale well. Make sure your individual call usually finishes in predictable worst case scenarios in 1/3 of the client’s response timeout. In our example we chose 20 seconds for timeout. Mind network latency and external script worst case runtime.

The solution here is decoupling:

Don’t make any UI element depend on live data from the worker
make a service/daemon or cron job collect worker state at short intervals and serialize these data points in time stamped files or directories
Create an api entry point to collect most recent state/results
Collect results of all workers from a commandline script, daemon, cron job or service in reasonable sequences.
Don’t expect most tasks immediately but add them to a queue. Horde_Queue may help you with that task.

Choose wisely where to call existing external apps and where to resort to PHP and the Horde Framework to solve common data collection, processing, formatting and returning tasks.

Remember to have fun.

The author is severly biased towards all things horde and has used horde classes and applications to solve various work-for-hire problems. The Horde Framework is one of the oldest and mature php projects and drives mission critical collaboration and data retrieval software all over the globe.

Horde Project pushes libraries to the frontpage

Posted on August 30, 2011 by Ralf

As you might know, the horde project does not only release a set of production quality software (and an interesting bunch more which are not yet release quality) but also provides over 80 well-designed loosely coupled libraries which help you build websites, business applications or even commandline tools. To stress that point, the Horde Project now put a link to the list of components right on their frontpage. Use Horde_Rdo, a lightweight ORM layer or use the RFC-compliant Imap_Client library which performs equally or even better compared to PHP’s interpreter extension written in c. Horde_Auth, a versatile and pluggable authentication layer, has recently been featured in a series of blog posts by lead developer Jan Schneider.
Like in Symfony or Zend Framework, Components are released along with a PHPUnit based test suite adapted in the Horde_Test class and can be obtained individually through the Horde Pear Channel.

Jan Schneider: Automatic twitter messages with Horde_Service_Twitter and two lines of code

Posted on June 15, 2011June 15, 2011 by Ralf

Jan Schneider just posted a damn cool use case for the Horde_Service_Twitter library. Using this library, just a few lines of php code are enough to send messages to your twitter stream like this:

#!/usr/bin/env php
<!--?php
/* Keys - these are obtained when registering for the service */
$keys = array(
'consumer_key'        => '',
'consumer_secret'     => '',
'access_token'        => '',
'access_token_secret' => ''
);

/* Enable autoloading. */
require 'Horde/Autoloader/Default.php';

/* Create the Twitter client */
$twitter = Horde_Service_Twitter::create(array('oauth' => $keys));

/* Send tweet */
try {
$twitter->statuses->update($argv[1]);
} catch (Horde_Service_Twitter_Exception $e) {
$error = Horde_Serialize::unserialize($e->getMessage(), Horde_Serialize::JSON);
echo "$error->error\n";
exit(1);
}

Now that’s neat, isn’t it.
In another team I worked with, we used a perl library which sent jabber/xmpp streams to our chat accounts when something ran into an uncought exception.

This might be worth porting to PHP/Horde some day.

Horde 4.0.6 brings user-specific admin privileges

Posted on June 15, 2011June 15, 2011 by Ralf

Traditionally, Horde only knows two kinds of users: Users with administration flag and users without. The list of admins is a static entry in the horde config file. It’s all or nothing – either a user gets access to all admin functions or to none. At least until recently.

Last October I wrote about a patch for Horde 3 which allows permission-based access to individual admin privileges. This patch has now been ported to Horde 4 and is incorporated in Horde 4.0.6. You can now assign a user the task of managing groups without allowing him to use the permissions admin and grant himself additional privileges. Or you can delegate emergency password resets to a group of trusted people without confusing them with icons like the PHP Shell. Only those admin functions are shown which the user has access to. Another side effect: Even if a user has all admin permissions, he is still not recognised as an admin and won’t be shown things that admins always have to see regardless of their permissions and settings.

In theory, you can now give yourself all admin permissions and safely delete yourself out of the admin list – as long as you have the “configuration” permission, you can always go back and restore without manually editing the conf.php file.

The Administration permissions are handled in the permissions screen just like any other user permissions. They live under the “horde” component. Currently only the “show” flag is actually recognized but this will be expanded later.

Migrating Horde 3 to Horde 4 – Top 6 ways to mess up

Posted on June 1, 2011June 1, 2011 by Ralf

There have been some migrations of Horde 3 to Horde 4 recently – not all went smooth from the start.

Some top issues of messing things up and how to avoid it:

initial application dimpIn Horde 3 dimp was a separate application which provided an ajax interface to imp. It has since been merged into the imp application. If your Horde installation had dimp before migration to Horde 4, this might create runtime issues for your users when
- when you locked the initial application to dimp
- when your users decided that their initial application should be dimp
To get around this you should
- make sure you didn’t blindly copy your locked settings from horde 3 to horde 4
- run a mysql update statement on the horde_prefs table to update column pref_value to “imp” if it was “dimp” before (Consider hiring a professional admin for the migration if you don’t know how that looks like)
Making changes in backends.php or prefs.php
In Horde 3 admins used to edit prefs.php or backends.php/servers.php to change values. Horde 4 ships backends.php and prefs.php as default values. Admins are supposed to copy these to backends.local.php and prefs.local.php and make their changes there. Changes to the original files will be overridden with the next rpm or pear update of the horde apps.
Not unchecking utc time in kronolith
The Horde 3 Calendaring app defaulted to store calendar events in local user time. The Horde 4 default is UTC timestamps. If you migrate from horde 3 you either have to uncheck that setting or run a migration script on the data.
Warning: You might end up with an unrecoverable state if you add new data in UTC mode to a calendar backend which has not been converted to UTC timestamps
Not converting turba and kronolith databases to utf8
In Horde 3 installations, the calendar app kronolith and the addressbook turba often had their database tables encoded in latin1. The system wide default in Horde 4 is utf8. Not adapting this setting to the tables or the tables to this setting results in corrupted display of international characters and symbols.
Warning: You might end up with an unrecoverable state if you add new data to addressbooks or calendars where backend encoding does not match the set horde encoding
Relying on menu.php’s javascript onclick handler or target attribute
In the ajax views of kronolith and imp there is currently no support for the target and onclick handler attributes. I do not know of any plans to re-add this support. If you want to link external sites into the iframe, consider creating a custom portal block. There was a recent blog post on creating this kind of blocks on The Upstairs Room
Using the ldap prefs backend
The ldap backend for preferences is currently not yet ported to horde 4. If you want to migrate, you first have to extract your prefs from ldap and then convert them to sql. If you need ldap prefs, consider hiring a consultant or sponsoring the development of this feature.

Tres consejos principales para ejercicios de entrenamiento de culturismo nvrzone ranbaxy drogas en venta mejor mezcla de musica ncs gym workout.

Horde 4 submit-requested into OpenSUSE 12.1

Posted on May 26, 2011 by Ralf

Today I submit-requested the Horde 4 Application Framework and the stable apps for openSUSE Factory.
This is becoming openSUSE 12.1 if the packages get accepted on time. They are currently in review.

openSUSE Legal team wants to review all packages’ licensing – I’m sure that’s NOT the fun part of their job.

If everything works fine, openSUSE 12.1 will be the first distribution to feature horde 4 in their mainstream repositories.

Continue reading “Horde 4 submit-requested into OpenSUSE 12.1”

Horde Team announces RC1 of Horde Groupware 4.0

Posted on May 18, 2011May 19, 2011 by Ralf

Since the release of Horde 4 and select applications, people kept asking for a new release of the popular Horde Groupware and Horde Groupware Webmail Edition bundles.

Horde is now about to release such a bundle, which features new ajax frontends for calendaring, tasks and improved mobile frontend for mail.

The new bundles will be released as pear collections. Gone are the tarballs.

Horde 4 Alpha 1 released (pear)

Posted on March 9, 2011March 9, 2011 by Ralf

Yesterday the horde project released alpha versions of Horde Framework 4 and the Groupware apps (Notes, Calendar, Email, Filter,Address Book, Tasks)

I did a test drive and they basically work. IMP has been improved a lot and now integrates the mobile and ajax interface versions which came as separate apps in Horde 3. DIMP (Ajax version) now plays more nicely together with classic non-Ajax horde applications.

I will begin distribution packaging for SUSE Linux around the official release on April 05, 2011.