PHProjekt V6 kicks out of the door on March 18th


PHProjekt V6, our little beastbee and Open Source project management software, kicks out of the door on March 18th as GA. It’s a complete rewrite (and thus, a major version update was needed) with Zend Framework and dojo and now licensed under LGPL (earlier versions were licensed under GPL). You may ask what you want to do until March 18th? Well, besides eating popcorn for Film@11:

 

 

The bee is watching you!

A comet over PHProjekt 6

So far PHProjekt 6 (P6, see http://phprojekt.com) is already enhanced with nice AJAX workflows and snappy user-experience. Nevertheless, we discussed a way to provide synchronous communication and direct information within the application.

Everybody knows GoogleMail with its easy to use frontend. Maybe you use it for your daily work. In GoogleMail, there is no need to refresh the page to receive a new mail, Google informs you automatically whenever a new mail is available. But how is this possible? The answer to this question is really simple: The server triggers a signal informing that a new mail is available. This technology is called Comet and describes a way how the server communicates with the client [see http://en.wikipedia.org/wiki/Comet_(programming) ].

Is there a way to use Comet for P6? As P6 works with a lot of users, it is important for me as a user to be informed when somebody has changed something in my projects or has added an urgent todo. The list where a notification could be triggered is long, but at the moment, only email notifications are sent immediately. So, what we need in P6 are real-time notifications. With real-time notifications, for example, every time somebody changes something in my project, I receive a message in form of a highlighted info box. This looks like a typical use case for using Comet. And yes, Comet would be great for realizing this kind of real-time notification! But since the users love the handy system requirements (Apache Server and MySQL) and the ease of installation, something different is needed than a real Comet architecture, because all the Comet magic is based on a so called application server, e.g. Jetty, Persevere. Therefore, I decide to implement another (Comet-) technology, called Long Polling. You can find the name AJAX Polling for this technology, too.

The technology behind Long Polling is to open a connection from the client to the server and not to close this connection immediately. Ok, that´s fine, but how to open a connection and „hold“ it? Opening a connection to the server is easy, a simple AJAX will do the trick. P6 uses the Dojo Toolkit for all its AJAX, so in this case I used the Dojo.xhrPost.

 

Since the HTTP protocoll is connection-based, at the server-side, a process is also needed that holds the connection and does not return . So, what I do to hold this connection open is to set a simple timeout during the execution of the server-side script, in my case I use sleep(). This is the basic idea of long polling, but the gimmick of this approach is to return earlier if something happens on the server-side. In other words, I open a connection to the server and check whether there are any changes, if not, the connection stays open for the maximum polling time. If yes, the response will be returned to the client immediately.

This diagram shows the basic workflow, and the lines below show the possible solution in PHP:

 

For P6 this means that I always save the following: the person who triggers an event, the event itself (e.g. somebody adds a new note), the item, the project, the creation time, the time until this message is valid, and the persons who should receive a frontend message from the database. The long polling loop checks whether there is anything new. If yes, it returns the data. If not, the polling loop starts again. Every event is saved in the database for a maximum of two minutes, except events in the future. Itis real-time – so itis possible to miss a message ;-)

One word about the Dojo Toolkit. The AJAX API of Dojo provides several functions to communicate with the server. The easiest way is to use dojo.xhrGet or dojo.xhrPost. I decided to use the Post version. Although the function has a property setTimeout, which allows you to set the time to wait for a response from the server, this is not needed, because of the sleep loops at the server side.

Now, let’s look at the downsides of the long polling approach. Yes, I partly agree with everybody who says that the permanent polling to the server causes massive traffic and server load, especially with an increasing number of users. But in the special case of P6 this is OK, because the number of users is manageable and in most cases, P6 runs in a local network. In addition, the frontend messages are configurable, You can set the polling loop and the number of requests to the database.

Considering all the facts and keeping in mind that P6 is designed to be a lightweight open source groupware, I am positive to say that the Long Polling technology is the right choice for P6.

Mayflower LDAP at Barcamp

At the Barcamp 2009 Alex, Michele, Michel, Markus, Sven and I continued a project that had already begun at the Barcamp 2008 – Creating an LDAP structure for internal Mayflower use and implementing a basic tool set for the administration.

A new LDAP scheme was created, a maintenance tool based on Zend Framework and Dojo was created and the tool was embedded as addon in the current PHProjekt 5

Weiterlesen

PHProjekt 6 Developer Preview edition

Today the PHProjekt core team is glad to publish a first edition of PHProjekt 6, the ‚developer preview release‘. More than one year has passed since the first commit into the SVN. In our view, the most important feature of the new version is the state-of-the-art architecture, which  makes a huge difference to all previous versions of PHProjekt.

With the choice of Zend framework in the backend and a complete integration of the dojo toolkit for the frontend, we think that the application is now ready for complex requirements and workflow processes. The MVC and active record pattern support scaffolding, thus allowing an effective and rapid development for new modules and features – in fact, new modules can be built with some mouse clicks.

Furthermore we tried to provide you a software, which makes fun to develop own code.
PHProjekt 6:
– is completely php5-OOP, well structured
– follows php5 coding standards (actually zf coding style)
– has a pretty good code coverage with unit tests
– is completely documented in phpdoc
By following ‚convention over configuration‘, many modules and services show an expected default behaviour. The API and protocol (JSON) between frontend and backend allows a change of single components of the application.

For other webmasters without coding skills, we tried to facilitate the application management with a WYSIWYG module designer.
Last but not least, we changed the licence from GPL to LGPL to give you more flexibility for the distribution of your own modules and extensions.

Lots of changes have been made over the past months by the developer team so we are really looking forward to get feedback from the community.
You can download the developer preview release here. We reopened the phprojekt repository on sourceforge, so you can checkout the source code from SVN there.
On phprojekt.com, we created a new section for PHProjekt 6, which we will fill with content in the next weeks. You can leave general remarks as a comment on this blog entry. For questions and discussions, we installed an own developer mailing list, please subscribe here. Anyone of you who want to inspect it directly, we have a setup a sandbox.
Please keep in mind, that it is only a preview, which means that something weird can frequently happen :-)
Please post your opinion, report bugs or send patches, everyone is welcome. We need you, the people who use it, your testing and reports, and your patches.

We wish you merry christmas … sorry: a lot of fun with the preview of PHProjekt 6!
the PHProjekt development team

PHProjekt in Poland – Meeting with deputy prime minister

Sometimes it is great to work for open source projects:

Monday and Tuesday I was in Warzaw on the invitation of the deputy prime minister, Waldemar Pawlak. The topic of our meeting was the strategic use of PHProjekt as the project management tool in his Ministry of Economy and a possible indroduction as collaboration platform among several governmental institutions.

I was surprised by his proficient skills in PHProjekt (he has been using it since version 3) and the deep knowledge of Open Source and LAMP applications. He pointed out the importance of Open Source in the EU regarding the free movement of services and information.

It’s nice to see that in many European countries the Open Source idea gets more and more visible in the public (and private) sector!

1. Mayflower Barcamp

Letztes Wochenende fand das erste Barcamp von Mayflower statt. Alle Mitarbeiter aus allen Standorten von Mayflower und SektionEins trafen sich in einem idyllischen Bauernhof im Altmühltal. Die Zielsetzung wich von einem traditionellen Barcamp etwas ab: Teams fanden sich zusammen, um an einem selbstgewählten Thema zu coden. Einzige Bedingung: Open Source oder Mayflower-Kontext! Weiterlesen

Dojo 1.0 released

Gestern wurde endlich die
Version 1.0 von Dojo released (http://dojotoolkit.org). Im Vergleich zu Dojo
0.43 hat sich in Version 1.0 einiges getan, die wichtigste Änderung: es werden
nun die Funktionalitäten sauber getrennt in drei kleinere "Projekte".
Persönlich freue ich mich, dass in Sachen Performance sich einiges getan hat und
die Engine massiv beschleunigt wurde. Bereits in PHProjekt 5 haben wir Dojo für
einige Teile eingesetzt, für die Version PHProjekt 6 nutzen wir nun neben einigen
Widgets auch die Möglichkeiten des Event- Handlings und der DOM- Manipulation
von Dojo, sowie natürlich die Ajax- Funktionalität. In PHProjekt 6 wird eine
MVC- Architektur auch auf Seiten des Clients umgesetzt. Unabdingbar für die
Entwicklung des Client- MVC ist für mich ebenfalls die Tatsache, dass Dojo ein "normales" objektorientiertes
Interface für JavaScript zur Verfügung stellt, das Klassen und Vererbung
anbietet. Um es in den Worten von Kollege Hartmann auszudrücken: „Dojo
ermöglicht dadurch ernsthafte Programmierung mit JavaScript!“

Leider ist auch mit dieser Version das Problem der Dokumentation nicht endgütig verbessert worden, es
bleibt einem nicht erspart, sich durch das „Dojo- Book“ auf den Webseiten zu kämpfen. Hoffentlich gibt sich das im laufe der Zeit, denn das ist immer noch
ein großes Manko.

PHProjekt Version 5.2 now available

MAYFLOWER announces that Version 5.2 of its
free popular open source groupware suite “PHProjekt” is available
immediately and ready for download.

With the new version a foundation is made for a clearly
arranged and intuitive usable PHProjekt. This is made by a refactoring
of the user interface and the integration of the Dojo JavaScript
Toolkit, which enables an effective realisation of Web 2.0
functionalities. The Migration is going to take place in steps, the
main changes of the user interface will take place in the next
releases. The clou: PHProjekt remains barrier-free.

Weiterlesen

An alternative Approach to Tagging

The Term Tagging

The popular feature of ‚tagging‘ content is nothing new. The average
netizen should have encountered it by now. Tagging was made popular
by sites like del.icio.us and flickr, where users can attach free-form
strings, so-called ‚tags‘ to their bookmarks and images. The viewer can
then use these tags to navigate through one or more user’s contents and
locate related content.

Scientific Background

The most-cited work on tagging is this research paper from HP, which
starts categorizing tagging as ‚folksonomy‘ (folk taxonomy) in contrast
to the conventional term taxonomy. A taxonomy is usually a categorization
of content according to a hierarchical and exclusive tree of attributes,
while the folksonomy is neither. Also, usually a taxonomy is created by
an individual or a small group, whereas the folksonomy can be created
and expanded by any number of users.

Use of tagging

The most obvious use of tags attached to a set of items is to be able
to quickly search for a subset without having to care about spelling
or being limited because the items are only indexed by title. The most
common user interface for searching according to tags is the ‚tag cloud‘,
a cluster of tags sorted alphabetically. For ease of use the font size
in which each individual tag is shown corellates to the number of items
it is attached to.

It is quite convenient to search in a tag cloud with a couple of hundred
items. A drawback of the cloud, though is that one can only search for
one tag at a time. When the search hits are shown, a list of related
tags is shown that share at least one item with the chosen tag. The
user can then proceed to view the items attached to one of these tags,
but the first tag is subsequently forgotten.

To understand why tagging was invented it is neccessary to look at the
prior status quo of hierarchical ordering like it is done in directories
like dmoz.org, ISP portals or most probably in your browser’s bookmarks
feature. Items are sorted into
a kind of taxonomy. This leads to difficulties since the universe of
topics is multidimensional.
One might have a folder ‚Tutorials‘ for sites that contain tutorials,
and a folder ‚Software‘ for sites on software. Consider further, that
he might want to divide the bookmarks between ‚Windows‘ and ‚Linux‘. The
hierarchical model now makes it necessary to duplicate one set of the
tree’s nodes to attach them to both branches, i.e. you either have a
first-level dichotomy between Windows and Linux and two second level
ones between Tutorials and Software or the other way round.
Since there’s more aspects to expect, this is not a feasible solution,
not even if you flatten out the hierarchy by combining the two aspects
and create four folders.

The Alternative: Instant Hierarchy

A method that combines the flexibility of tagging with the
search-narrowing power of a deep hierarchy is to combine the tags to an
‚instant hierarchy‘: The user chooses from a pool of keywords. Like in
tag clouds, he gets to see some items then and a list of subsequient
keywords. He can then choose a second keyword and get the items that
are tagged with both chosen keywords, and so on. This instant tree is
at it’s deepest in the path of the item with the most tags.




These four images show an example of navigation through the instant
hierarchy. The choice of keywords in the selectboxes narrows downn, in
each step the items that match the exact set of keywords are shown. The
best results are achieved when the items are tagged diversely and
with a diverse number of keywords, because every step then show some
information. If all items have, for instance, three keywords, the first
two steps show nothing new.

Database-wise, this approach is a lot more complicated than the
single-tag navigation of del.icio.us or technorati. Depending on the
database available, n-fold joining or subquerying can be used. In both
cases we start with a set of three tables, ‚bookmarks‘, ‚keywords‘ and
‚relations‘ since we want to map an n:m relationship:

The ID columns are the primary keys (although the relations table might
not need an extra key), the keywords should be unique anyway as long as
no ownership is implemented.

N-fold Joins

The query to retrieve the first set of keywords is obvious:

SELECT DISTINCT k.id,k.keyword
FROM relations as r, bookmarks as b, keywords as k
WHERE r.keyword=k.id
AND r.bookmark=b.id

That might look complicated, but it easily sorts out the keywords that have nothing attached, and it fits in with the recursion that follows
The second step looks like this:

SELECT DISTINCT k.id,k.keyword
FROM relations as r, bookmarks as b, keywords as k, relations as r0
WHERE r.keyword=k.id
AND r.bookmark=b.id
AND r0.keyword=5

since the id of keyword „Programming“ is 5 (you can do this with another join of course, but the information has to retrieved to get the bookmarks beforehand)
The third step is where it gets complicated – we need to make sure the database looks for both keywords and does not return either

SELECT DISTINCT k.id,k.keyword
FROM relations as r, bookmarks as b, keywords as k, relations as r0, relations as r1
WHERE r.keyword=k.id
AND r.bookmark=b.id
AND r0.bookmark=r.bookmark
AND r.keyword!=5
AND r0.keyword=5
AND r1.bookmark=r.bookmark
AND r.keyword!=7
AND r1.keyword=7

From there, it’s simply recursion.
The queries for the list of bookmarks are similar, but a bit simpler. The fourth list of bookmarks is retrieved with this query:

SELECT DISTINCT b.ID,b.url,b.title
FROM bookmarks as b, relations as rc, relations as r1, relations as r2, relations as r3
WHERE rc.bookmark=b.id
AND r1.keyword=’5′
AND r1.bookmark=b.id
AND r2.keyword=’7′
AND r2.bookmark=b.id
AND r3.keyword=’12‘
AND r3.bookmark=b.id
GROUP BY b.id

Subqueries

Newer MySQL versions offer the use of subqueries to replace some joins. A query to get a list of keywords with a prior choice looks like this:

SELECT DISTINCT k.id,k.keyword,
(SELECT count(r0.keyword)
FROM relations r0
WHERE r0.bookmark=b.id
AND r0.keyword IN (5,7,12)) x
FROM bookmarks b,
relations r,
keywords k
WHERE k.id=r.keyword
AND b.id=r.bookmark
AND k.id not in (5,7,12)
HAVING x=3
ORDER BY k.id

We retrieve all keywords that have x other keywords attached, where the other keywords are chosen from our list.
In the same way we can subselect and count the attached keywords to retrieve an appropriate list of bookmarks here.

Implementation

I assembled a small Addon for our OpenSource software PHProjekt and
plan to release it on PHProjekt.com as soon as it’s presentable
(it’s a hobby project so it proceeds a little slower than I would wish
it to). If you are interested in a snapshot, drop me a mail (brotzeller at mayflower dot de).
Currently it lets the user add and delete keywords and bookmarks
and assign keywords to the bookmarks. The above screenshots show the
navigation, which can also return xml and can therefore be used as sidebar
in gecko browser. I also added a feature to get RSS feeds for keyword
choices, so firefox users can add dynamic bookmarks. Since it’s about
bookmarks, usage is observed via redirection, so the RSS feeds can also
show new or most used or last used bookmark lists.

Announcing release of PHProjekt V5.0.2 and the beta release of V5.1

Wow! Our development team did a terrific job baking some new releases this week. While Albrecht is enjoying his holidays very much, I can feel the sweat of the team preparing everything for launching in time two releases.

The 5.0 branch got a new release: 5.0.2. It mainly is a bug fix release and marks the last release of the 5.0 branch which will be closed as of today. Just hang over to the release announcement by Franz Graf on phprojekt.com.

As of today, we are proud to announce the release of the first beta version of PHProjekt V5.1. Our Mayflower fellow Nina Schmitt tells: „Our main goal was to further enhance the usability and functionality of PHProject with this new Version, but some new features were added as well“. She points out that we need further tests from the community and report bugs on the bug tracker at our thinkforge.org development forge.

Continue to read this post to see what is new in this 5.1 branch. And don’t forget to download it!

Weiterlesen