Kraft provides effective invoicing and document management for small businesses on Linux. Check the feature list.

This new version is a big step ahead for the project. It does not only deliver the outstanding ports to Qt6 and KDE Frameworks 6 and tons of modernizations and cleanups, but for the first time, it also does some significant changes in the underlying architecture and drops outdated technology.

Kraft now stores documents not longer in a relational database, but as XML documents in the filesystem. While separate files are more natural for documents anyway, this is paving the way to let Kraft integrate with private cloud infrastructures like OpenCloud or Nextcloud via sync. That is not only for backup- and web-app-purposes, but also for synced data that enables to run Kraft as distributed system. An example is if office staff works from different home offices. Expect this and related usecases to be supported in the near future of Kraft.

But there are more features: For example, the document lifecycle was changed to be more compliant: Documents remain in a draft status now until they get finalized, when they get their final document number. From that point on, they can not longer be altered.

There is too much on the long Changes-List to mention here.

However, what is important is that after more than 20 years of developing and maintaining this app, I continue to be motivated to work on this bit. It is not a big project, but I think it is important that we have this kind of “productivity”-applications available for Linux to make it attractive for people to switch to Linux.

Around Kraft, a small but beautiful community has built up. I like to thank everybody who contributed in any way to Kraft over the years. It is big fun to work with you all!

If you are interested, please get in touch.

This is the release announcement of the new Kraft version 1.2.2. This is a small service release that fixes a few bugs and CI issues.

Right after this release, the branch with significant changes for Kraft 2.0 will be merged to master. These changes will make Kraft ready for sharing documents across private file clouds and with that enable use cases for distributed use via internet, along with other significant feature updates.

Details about the next big release with version number 2.0 can be read on the Github Discussion page.

Any feedback and contribution is highly appreciated.

It was never properly released, so recently I decided to wrap it up and finally cut a first release which people can find on the release page on Codeberg.

Technically it uses the XSLT stylesheets provided by Kosit and calls an external java process on the local machine to run that through a specific Saxon processor. For that, XRView requires a java runtime installed.

Since the setup of these dependencies is a bit cumbersome, the new release 1.0 does that for users. It downloads the stylesheets and also the saxon processor runtime from their upstream repositories and stores them on the local machine for future use. Of course it is strongly recommended to double check the downloaded resources for their validity and integrity and not to run software that some other code downloaded.

More new features in this first release are:

• internationalization, first available language is German
• a rudimentary application menu with about dialog and such
• the setup routine as described above

Note that this is the first release of the software. Yet, I think it is useful, and a interesting starting point for further activities in this area. As XRechnung will become a mandatory standard for all companies in Germany (at least) I think it is very important to have a free software alternative. There are already many commercial offerings.

However, I am not feeling to develop and maintain this as an “one man show” forever. Being kind of frustrated about the way how free software is often consumed nowadays, I will happily continue to contribute to it if there is more interest than “gimme for free” by other people or organizations.

Let’s see if this is heading somewhere :-)

No, not yet?

Ok, well, in that case, here comes rescue: With the little script described here, it becomes really easy to start Infinite Scale to check it out on your computer. It makes it really easy and quick for you, without Linux super admin powers whatsoever.

To use it, you just need to open a terminal on your machine and cd into a directory somewhere in your home where you can afford to host some bytes.

Without further preparation, you type the following command line (NOT as user root please):

curl -L https://owncloud.com/runocis.sh | /bin/bash

What it does is that it automatically pulls the latest stable version of Infinite Scale from the official download server of ownCloud onto your computer. For that, it creates a configuration and a start script, and starts the server. The script detects the platform on which your’re running to download the right binary version. It also looks up the hostname and configures the installation for that name.

Once the server was started Infinite Scale’s web client can be accessed by pointing a browser to the URL https://your-hostname:9200/. Since this is an installation for testing purposes, it does not have a proper certificate configured. That is why your browser is complaining about the cert, and you have to calm it. And indeed, that is one of the reasons why you’re not supposed to use this sneak peak in production or even exposed to the internet.

For the nerds, the script does not really do magic, but just curls the golang single binary of Infinite Scale down to the machine into a sandbox directory, chmod it to be executable and create a working config and a data dir. All happens with the priviledges of the logged in user, no sudo or root involved. You’re encouraged to double check the install script using for example the command curl -L https://owncloud.com/runocis.sh | less - of course you never should trust anybody running scripts from the internet on your machine.

If the server is stopped by pressing Ctrl-C, it later can be started again by the script runocis.sh that was kindly left behind in the sandbox as well.

The installer was tested on these three platforms: 64 bit AMD/Intel CPU based Linux machines, 64 bit Raspberry Pi with Raspbian OS and MacOSX. The flavour of Linux should not make a difference.

If you encounter a problem with the script or if you have suggestions to improve, please find it in my this’n that section on Github. I am happy to receive issue reports or pull requests.

For further information and setups suitable for production please refer to the Infinite Scale documentation.

One of the topics is the VFS (Virtual Filesystem-) API for the Linux desktop. Indeed that is a topic for us at ownCloud as well, and I like to share our perspective on it, discussing it in the scope of the free desktop.

The topic is very important, as “syncing” of data from and to cloud storages has changed over time. From having all files mirrored from client to server and vice versa, it has now shifted to keep all files in the cloud, and have them as so called placeholders on the desktop. That means that most files on the client appear with size zero to save space, but the complete filesystem structure is available.

If a user starts to interact with such a dehydrated file, the content is of the file is downloaded transparently utilizing the cloud system client, for example ownClouds desktop client. The same happens when an application accesses such a file. As a result, the placeholders look and behave like the normal filesystem we are used to.

On Windows and on MacOSX, the problem is kind of solved. Both have added APIs to their OS that can be used to implement the access of data on the cloud.

On Linux, we do not have this kind of API yet. That means that it is close to impossible to implement this user experience. Volker already said that desktop environment specific solutions probably do not scale, which I agree with.

At ownCloud we have looked into the implementation of a specific FUSE file system. That should certainly be possible, and is probably a part of the solution, but is considerable effort because of the asynchronous nature of the topic. Given that the market share of Linux desktop systems is pretty small it is not attractive for companies to invest a lot into a Linux only system. Here the power of community could make a difference again.

It would be best if we as open source community would come up with a shared solution as a free desktop standard, that might be oriented on one of the existing APIs, maybe the MacOSX File Provider API: A library and little framework that the linux desktop environments can work with abstracting the VFS.

While collaborating on that, all data clouds could implement the bindings to their storage. With that, the extra implementation efforts for the Linux solution hopefully wouldnt be dramatic any more.

Let’s call this system openVFS as a work title. How can we evolve it? I’d like to invite all interested parties to discuss in this temporary Github repo to collect ideas and opinions. There is also a little experimental code.

Today we are releasing Kraft Version 1.1 with significant improvements for users and the Krafts integration with latest software such as cmake and KDE.

It received updated dutch translations in UI and also for the manual. The application icon was fixed, and some cmake related fixes were done that make Kraft working with different versions of Akonadi that are available on different distributions.

Macros

For users, two significant improvements are included: The header- and footer texts of the documents now may contain macros that support automatic computing of values such as dates that depend on the document date. With that, it is for example easy to have for example a payment date printed out on the document, that is ten days later than the document date.

There are even more interesting macros, stay tuned for a separate post about this feature.

Insert Templates Button

The second new feature is a new button that allows to insert templates for the header- or footer text at the cursor position. Before it was only possible to replace the entire text with a template. This will give users way more flexibility how to structure template texts.

In parallel to these improvements, work is also going on in a branch for Kraft 2.0 which will enable more collaborative functions for Kraft.

Many commercial vendors have already caught up and provide ways to generate XRechnung formatted documents with their software. However, to my knowledge, the availability of open source end user software is very limited. Since the standard itself is at least very open and transparently documented, so I think it is worthwhile to also support it with free software on the desktop.

Kraft, the desktop software for invoicing and efficient office work in the small enterprise, supports export of XRechnung documents since a while.

Over the weekend I created a new little project that adds a viewer for XRechnung documents called xrview.

A german city was looking for something to evaluate processes in a Linux- and KDE based productivity work environment.

Technically it is not very sophisticated: It renders the provided XML file using XSL styles officially provided by the Koordinierungsstelle für IT-Standards in a two step process to HTML, which is displayed in a web view. Some values of interest are extracted from the XML and displayed in a detail pane on the left side.

This is just a POC and has to be continued, but the time was good to kickstart this project.

Maybe anybody is interested to create a PR to help to improve digitalization in Germany?

These are only the most obvious points why Kraft is so far only available for Linux.

But often enough switching to Linux is another hurdle that users have to go, coming from a more mainstream world. So it is great to learn that Kraft, as many UI applications, can be run unter the Windows Subsystem for Linux (WSL) on Windows systems quite flawlessly.

It was tried with Windows 11 Home and the Kraft AppImage of 1.0.

The steps to run Kraft in WSL:

• Make WSL running on the Windows installation
• Install a X-server on the Windows machine
• Create a Debian or Ubuntu Linux subsystem
• Install a few extra packages into the Linux installation
• Download and run the Kraft AppImage 1.0

A more detailed howto and discussion can be found in the easy cash & tax forum. Thanks Thomas for bringing up the topic and providing the Howto!

What is Kraft?

Kraft is free software to create office documents like offers and invoices in an efficient way. It runs on the Linux desktop and suits small businesses of all kinds.

After countless releases with version numbers 0.x, Kraft finally goes with version number 1.0 with this release. It is in production at several companies since many years, and with this release it got many more improvements that make it a really mature product.

Highlights

Lets take a look on the highlights of of version 1.0.

AppImage Support

Installing software properly is still a challenge for many users. AppImage comes to the rescue there: Single file, easy to download and start with all dependencies included makes it easy for users. For the creator, it is just one-fits-all, which makes it easy to maintain.

The new Kraft AppImage was carefully reworked and has now really all dependencies included, has a complete own icon set and was tested thoroughly. It is ready for production now.

Giro Code

Kraft now can print the EPC QR Code on invoices easily. Users just have to configure the bank account data to use, and Kraft creates the QR code automatically to be included in the documentation template.

User Manual

The Kraft user manual got great improvements again. It comes with even more and improved text, with much more screenshots and improved translations.

This is a community contributions that is so important for Kraft.

Many other Improvements

There were many other improvements: New functionality such as the new day counter for the number cycle templates (allows to have a counter in the document number that resets to 1 every day) or new modes for the watermark functionality to complete the generated PDF documents plus much more details (See Changelog).

How it continues

Kraft is up and running - and maintained. The plan is to keep the 1.0 as a kind of LTS in a stable branch, that only gets urgent fixes for stability.

For master, I am planning to do a couple of more intrusive changes that will take a bit longer.

See the roadmap discussion here.

Curious?

I hope you got a bit curious now about Kraft. Feel free to check it out using the AppImage.

If you find it an useful addition to the Linux application ecosystem that enables more users for Linux we’d very much appreciate your contribution!

This is the successor of my previous blog on https://dragotin.wordpress.com.

After paying wordpress quite some money to get an advertise free blog I decided to get rid of that and have my own hosted blog where I do not have to pay for not having battle ships or girls underneath my articles. Yes, that is true: Readers sent me screenshots with this kind of images.

So I am starting this journey here with Hugo. Let’s see how that turns out :-)

Kraft is software for the Linux desktop to handle quotes and invoices in the small business.

This is a version packed with bugfixes and also new features. The most important fixes were in the area of the catalog handling: Based on bug reports from the community the catalog window was completely reworked. Drag and drop of items in the catalog, the sorting and reordering of items are now working properly and as planned.

Another big addition is the support of . XRechnung is an E-invoicing format more and more mandatory in the governmental area in Germany. We are very proud that Kraft is the first open source office tool that supports that standard in a user friendly way. All invoices can now also exported in the XRechnung-XML format.

Beside these two big improvements, there are lots of others. For example, the user manual was further improved and is available also in Dutch. A lot of other smaller but non the less important improvements and fixes make version 0.98 a valueable release.

We wish a lot of fun with this new improved version of Kraft!

The good thing about this quiet holidays is that there is time to work on projects. I had a few nice changes of PDF Quirk laying around, and now finalized them to a new version. Please welcome PDF Quirk version 0.95!

What is PDF Quirk?

PDF Quirk is a little desktop utility to create PDF files from images, targeted to non nerdy desktop users.

Sending PDFs is (still) often a requirement in offices where people are asked to transfer PDF files via email, or better by pushing them through their private ownCloud.

The source images can either be loaded from file, or directly scanned with an hardware scanner. For that, PDF Quirk utilizes the tool scanimage from the SANE Projekt, to avoid reinventing the wheel. Configured once, that works like a charm.

Having scanned or picked the source images, they can be deskewed, turned and rearranged, and finally converted to a good quality PDF file with reasonable file size.

New Version 0.95

The new version brings a few features and some fixes:

• Images can deskewed now
• Basic PDF options like margin, paper size and orientation can be set
• The UI got a little cleanup
• Translations were added, first new language is German
• Dependencies were reduced, new Qt based PDF generator was implemented
• Builds with Qt5 and Qt6

Interested?

Check out PDF Quirks Website for more information.

PDF Quirk is free software. Please contribute through the Github repository.

Fine with me. Just to support that, I did a little hackery over XMas to improve the support for ownClouds virtual file system on the Linux Desktop.

What are Virtual Files?

In professional usecases, users often have a huge amount of data stored in ownCloud. Syncing these completely to the desktop computer or laptop would be too much and costly in bandwidth and harddisk space. That is why most mature file sync solutions came up with the concept of virtual files. That means that users have the full structure with directories and files mirrored to their local machines, but have placeholder of the real files in the local file manager.
The files, however, are not on the disk. They get downloaded on demand.

That way, users see the full set of data virtually, but save time and space of files they never will need on the local system.

The ownCloud Experience

ownCloud has innovated on that topic a while ago, and meanwhile support virtual files mainly on Windows, because there is an elaborated system API to work with placeholder files. As we do not have this kind of API on Linux desktops yet, ownCloud desktop developers implemented the following solution for Linux: The virtual files are 1 byte sized files with the name of the original file, plus a suffix “.owncloud” to indicate that they are virtual.

That works, yet has one downside: Most file managers do not display these placeholder files nicely, because they loose the MIME type information. Also, downloading and also freeing up the space of downloaded files is not integrated. To summarize, there is a building block missing to make that useful on Linux.

Elokab-files-manager with ownCloud

I was wondering if it wasn’t possible to change an existing file manager to support the idea of virtual files. To start trying I found the Elokab-files-manager which is a very compact, yet feature rich file manager built on Qt. It is built with very few other dependencies. Perfect to start playing around with.

In my Github fork you can see the patches I came up with to make Elokab-fm understand ownCloud virtual files.

New Functionality

The screenshot shows the changes in the icon view of Elokab-fm.

Screenshot of patched Elokab-files-manager to support ownCloud Virtual Files.

To make that possible, Elokab-fm now pulls some information from the ownCloud Sync Client config file and connects to the sync client via local socket to share some information. That means, that the sync client needs to run to make that work.

Directories that are synced with ownCloud now show an cloud overlay in the center (1).

The placeholder files (2) which are not present on the local hard drive indicate that by showing a little cloud icon bottom right. However, other than before, they are displayed with their correct name and mime-type, which makes this already much more useful.

Files, which are on the local disk as the image (3) show their thumbnail as usual.

In the side panel (4) there are a few details added: The blue box on the bottom indicates that the file manager is connected to the sync client. For the selected virtual file (2), it shows an button that downloads the file if clicked which would turn it into a non virtual, local file. There is also an entry in the context menu to achieve that.

Status

This is just a proof of concept and a little XMas fun project. There are bugs, and the implementation is not complete. And maybe Jürgen’s idea of a FUSE layer to improve that is the better approach, but anyway. It just shows what is possible with virtual files also on Linux.

If you like that idea, please let us know or send a little PR if you like. We do not wanna fail providing our share on the year of the Linux Desktop, right? ;-)

Building it from my Github branch should be fairly easy, as it only depends on Qt.

For openSUSE users, I will provide some test packages in my home project on Open Build Service.

Whenever I tried to share my KDE screen, the screen became black, surrounded with a red rectangle as indicator for the shared area. The people who I shared with also just saw the black area, and also the mouse pointer as me.

The problem is described in a bugreport and there are two ways of solving it:

1. Enable compositing: The red indicator rectangle requires that the window manager supports compositing. KWin can of course do that, and with that enabled, sharing works fine including the red rectangle.
2. If compositing can or should not be used there is another workaround: As the bug report shows, renaming the file /usr/share/teams/resources/app.asar.unpacked/node_modules/slimcore/bin/rect-overlay so that it is not used by teams fixes it as well. Obviously you wont have the red rectangle with this solution.

That said, it is of course preferable to use an open source alternative to Teams to help these evolve.

Kraft Version 0.95

With version 0.95 we do a big step forward in the way of generating documents: Until now (more than fifteen years!) Kraft uses the ReportLab python library to create high quality PDF documents.

While this has served us well, it has always been cumbersome to adopt the template for users needs. ReportLab uses a XML format as the template which has a bit of a steep learning curve and is not really easy with syntax.

This changes now: From version 0.95, Kraft supports the cool project WeasyPrint. The principle remains the same: The document is built of a text based template which defines the look of the output document. That gets filled with the document data and gets converted to a PDF document. But unlike ReportLab, Weasyprint is HTML based. Many people know a bit of HTML and thus will have an way easier time to adopt the template to personal needs.

Apart from the ease of use, it is much more simple to do modern report design with Weasyprint, as it supports the wide range of CSS styling.

That is a great improvement, as adopting Krafts output to personal needs is much more intuitive now. For now, Kraft supports both rendering engines, but ReportLab based reports are deprecated now and support will end in future releases of Kraft.

Along with integration of Weasyprint the text templating library Grantlee was added, as it is the standard in the KDE/Qt world, well maintained and widely available. The ctemplate library which was used for that so far will also be deprecated.

The other great improvement in this release is that Kraft now has a user manual embedded which will give new users a guiding help. It will open in the browser once the user clicks on the menu entry in the help menu, also without internet connection. It was started by a community member and it will grow and improve over time.

As usual this new version also ships an amount of bug fixes and small improvements which can be found in the Changelog.

We wish all users big fun with this remarkable new version of Kraft.

It is a very good sign to have the topic of internet search in the news. It is a fact that one company has a gigantic market share in searching which is indeed a threat to the freedom of internet users. The key to be found in the web is the key to success with whatever message or service a web site might come up with, and all that is controlled by one enterprise driven by commercial interests. That should be realized by a broad audience.

The Open Search Foundation has the clear vision to build up an publicly owned search index as an alternative for Europe.

Geographical and Political Focus

The whitepaper talks about working on the search machine specifically for Europe. It mentions that there are search indexes in the US, China and Russia, but none rooted in Europe. While this is a geographical statement in the first place, it is of course also a political, because some of the existing services are probably politically controlled.

It is good to start with a focus on Europe, but the idea of a free and publicly controlled project should not be limited to Europes borders. In fact, it will not stop there if it is attractive because it might offer a way to escape from potentially controlled systems.

On the other hand, Europe (in opposite to any single European country alone) seems like a good base to start with this huge effort as it is able to come up with the needed resources.

Organization

The founding members of the Open Search Foundation are not very well known members of the wider open source community. That is good, as it shows that the topics around the free internet do not only concern nerds in the typical communities, but also people who work for an open and future proof society in other areas like academia, research and medicine.

On the other hand, an organization like for example the Wikimedia e.V. might have been a more obvious candidate to address this topic. Neither on the web site nor in the whitepaper I found mentions of any of the “usual suspects” or other organizations and companies who have already tried to set up alternative indices. I wonder if there have been discussions, cooperations or plans to work together?

I am very curious to see how the collaboration between the more “traditional” open data/open source community and the Open Search Foundation will be, as I think it is a crucial part to combine all players in this area without falling into the “endless discussion trap” while not achieving countable results. It is the question of building an efficient community.

Pillars of Success

Does the idea of the OSF have a realistic chance to succeed? The following four pillars might play an important role for the success of the idea to build the free search index of the internet:

1. Licenses and Governance

The legal framework has to be well defined and thought through, so that it will be resilient longer term. As we talk about a huge commercial potential to control this index, parties might wanna try to get into control of it.

Only a strong governance and legal framework can ensure that the idea lasts.

The OSF mentions in the whitepaper that it is one of the first steps to set this up.

2. Ressources

A search index requires big amounts of computing power in the wider sense, including storage, networking, redundancy and so on. Additionally there need to be people who take care on that. For that, there needs to be financial support for staffing, marketing, legal support and all that.

The whitepaper mentions ideas to collect the computing power from academia or from company donations.

For the financial backing the OSF will have to find sources like EC money, from governments and academia, and maybe private fund raising. Organizations like Wikimedia would already have experience with that.

If that will not be enough, the idea of selling better search results for money or offering SEO help for development will quickly come up. This will be interesting discussions that require the strong governance.

3. Technical Excellence

Who will use a search index that does not come up with reasonable search results?
To be able to compete with the existing solutions that even made it into our daily communication habits already, the service needs to be just great in terms of search results and user experience.

Many already existing approaches that use the Google index as a backend have already show that even with that it is not easy to provide a comparable result.

It is a fact that users of the commercial competition trade their personal data against optimal search results, even if they dont do that consciously. It is more difficult for a privacy oriented service, so this is another handicap.

The whitepaper mentions ideas on how to work on this huge task and also accepts that it will be challenging. But that is no reason to not try it. We all know plenty of examples where these kind of tasks were successful even though nobody believed that in the beginning.

4. Community

To achieve all the points a strong community is key factor.

There need to be people who do technical work like administering the data centers, developers who code, technical writers for documentation, translators and much more. But that is only the technical part.

For the financial-, marketing- and legal support there are other people needed, not speaking about political lobby and such.

All these parts have to be built up, managed and kept intact long term.

The Linux kernel, which was mentioned as a model in the whitepaper, is different. Not even the technical work is comparable between the free search index and the Linux kernel.

The long term stable development of the Linux kernel is based on people who work full time on the kernel while being employed by certain companies who are actually competitors. But on the kernel, they collaborate.

This way, the companies share cost for inevitable base development work. There differentiators in the market are not depending on there work on the kernel, but in the levels above the kernel.

How is that for OSF? I am failing to see how enough sustainable business can be based on an open, privacy respecting search index so that companies will be happy to fund engineers working on it.

Apart from that, the kernel has the benefit that it had strong companies like RedHat, SUSE and IBM who pushed Linux in the early times, so no special marketing budgets etc. were needed for the kernel specifically. Also that is different for OSF, as quite some marketing- and community management money will be required to start.

Conclusion

Building a lasting, productive and well established community will be the vital question for the whole project in my opinion. Offering a great idea, which this initiative is without question, will not be enough to motivate people to participate long term.

There has to be an interesting offer for potential contributors at all levels, starting from individuals and companies for contributions, to universities for donating hardware or the governments and the European Community for money. There needs to be some kind of benefit they will gain for their engagement on the project. It is interesting if the OSF can come up with a model that will get that kickstarted.

I very much hope that this gets traction as it would be an important step towards a more free internet again. And I also hope that there will be collaboration on this topic with the traditional free culture communities and the foundations there.

For this very common use case I could not find a tool to make that really easy for the Linux desktop. Given my mission to help making the Linux desktop more common in the small business world (do you know Kraft?) I spent some time starting this little project.

Please welcome PDF Quirk, the desktop app to easily create PDFs out of images from storage or directly from the scanner!

It is just what this screenshot shows: A one page app to pick images from either the harddisk or from a scanner if configured, and save it right away to a multi page PDF file. The only option is to have either monochrome or color scan. No further scan chi-chi, just nice PDFs within seconds.

Of course I did not want to spend too much time and reinvent the wheel. PDF Quirk uses ImageMagicks convert utility and the command line scan client scanimage of the SANE Project in the background. Both are welknown standard commands on Linux, and serve well for this purpose.

Maybe you find PDF Quirk useful and wanna try it. You find it on Github, or packages on the openSUSE Buildservice.

Contributions and comments are very welcome. It is a fun little project!

Followup Documents

A great new feature is the completely reworked handling of so called follow-up documents. Kraft has this feature to help dealing with docs that follow on a certain kind of predecessor doc in the business flow, such as an invoice follows on an offer once the work has been finished.

To achieve that and to have all the items of the offer again referenced in the invoice, the Kraft user justs clicks on the menu item “Create Followup Document” on the selected offer, and the new doc is created with all the items of the offer prepared. It is ready to be adjusted to the final invoice details.

Now with version 0.90 this has even improved. Often, the flow is a bit different because a partial invoice is sent while the job is still in progress, for example to cover cost for material. A partial invoice usually has only one item, which is the progress payment the customer should pay.

Kraft users can handle that now very easy: The new UI does not only allow to choose from which predecessor document the final invoice should copy the items (which usually is the offer in the beginning) but it also realises that a partial invoice was sent before, so this amount of payment has to be substracted from the final invoice. Kraft adds an item to the final doc automatically here.

This helps to create high quality documents for your customers even faster, which is exactly the mission of Kraft.

Other Improvements

In addition to that, as usual a lot of other visible and invisible changes have gone into this release. For example, the UI has been simplified again by a clean up of the menu for example. The About Kraft dialog was removed and integrated into the Kraft information page.

Under the hood, we finally added unit tests, which is just a start so far, and ntroduced a new, XML based system to manage the available document types which makes igrations way easier. Last but not least the python based pdf conversion script was ported to python 3 to be ready for the end of python 2.

Community

Last but not least, Kraft got new contributions from the community: A new document type was initiated by a community member, which is a offer like document without showing the price information. Also, the complete app was translated to Dutch by a volunteer, and the translation is now shipped with Kraft by default. Also, other smaller bugfixes went in.

Thank you all for your help to improve Kraft.

Some people think it is a great idea to include the functionality in the sync client.

I don’t agree because it combines two very complex topics into one code base and makes the code difficult to maintain. The risk is high to end up with a kind of code base which nobody is able to maintain properly any more. So let’s better avoid that for ownCloud and look for alternatives.

A good way is to use a so called encrypted overlay filesystem and let ownCloud sync the encrypted files. The downside is that you can not use the encrypted files in the web interface because it can not decrypt the files easily. To me, that is not overly important because I want to sync files between different clients, which probably is the most common usecase.

Encrypted overlay filesystems put the encrypted data in one directory called the cipher directory. A decrypted representation of the data is mounted to a different directory, in which the user works.

That is easy to setup and use, and also in principle good to use with file sync software like ownCloud because it does not store the files in one huge container file that needs to be synced if one bit changes as other solutions do.

To use it, the cypher directory must be configured as local sync dir of the client. If a file is changed in the mounted dir, the overlay file system changes the crypto files in the cypher dir. These are synced by the ownCloud client.

One of the solutions I tried is CryFS. It works nicely in general, but is unfortunately very slow together with ownCloud sync.

The reason for that is that CryFS is chunking all files in the cypher dir into 16 kB blocks, which are spread over a set of directories. It is very beneficial because file names and sizes are not reconstructable in the cypher dir, but it hits on one of the weak sides of the ownCloud sync. ownCloud is traditionally a bit slow with many small files spread over many directories. That shows dramatically in a test with CryFS: Adding eleven new files with a overall size of around 45 MB to a CryFS filesystem directory makes the ownCloud client upload for 6:30 minutes.

Adding another four files with a total size of a bit over 1MB results in an upload of 130 files and directories, with an overall size of 1.1 MB.

A typical change use case like changing an existing office text document locally is not that bad. CryFS splits a 8,2 kB big LibreOffice text doc into three 16 kB files in three directories here. When one word gets inserted, CryFS needs to create three new dirs in the cypher dir and uploads four new 16 kB blocks.

My personal conclusion: CryFS is an interesting project. It has a nice integration in the KDE desktop with Plasma Vault. Splitting files into equal sized blocks is good because it does not allow to guess data based on names and sizes. However, for syncing with ownCloud, it is not the best partner.

If there is a way how to improve the situation, I would be eager to learn. Maybe the size of the blocks can be expanded, or the number of directories limited? Also the upcoming ownCloud sync client version 2.6.0 again has optimizations in the discovery and propagation of changes, I am sure that improves the situation.

Let’s see what other alternatives can be found.

For that I was looking for a so called boom box to stream to from my mobile, simple, dirty and loud. Good that I was a proud awardee at the HiFiBerry Maker Contest 2017 with my TeakEar build, where I won a nice set of a RaspberryPi Zero with a little HiFiBerry MiniAmp, coming with all what is needed to make that working. That should be enough to get proper punk sound in the workshop - and escape the boring normal commercial boom boxes all around.

At a flee market I found a great case for that - exactly one of these that were in the classrooms of my school giving the unbanning gong at the end of each lesson. Nicely with the original non-color textile hiding the speaker and nicely done bended and veneered wood for the elegance.

A great fit for my usecase.

Two 3.3 inch Visaton fullrange speaker are mounted in simple closed chassis. The Raspi with the Amp is mounted between them. All is mounted on a back plate that fits the school speaker chassis. Boombox ready.

From a software perspective, it is there is just a Raspbian running that is configured to act as a bluetooth audio device for my mobile.

I think it is a nice addition to my workshop, with a great ease of use since it auto-connects to my mobile. Does it sound great? Oh no, not ..really. Is it loud? Well, yes, loud enough to not get in trouble with the neighbors. Given it’s size, it is actually impresse.

Surely Lemmy would be fine to play Rock’n Roll through it :)

The only downsize of the whole thing is that it disturbs the radio reception quite a bit, so it is really either or. Any hints how to reduce that?

[caption id=“attachment_943” align=“alignright” width=“338”] Code lines changed as of Nov. 10, 2018[/caption]

That motivated me to do some investigation what the Nextcloud client actually consists of (at due date Nov. 10, 2018). I was looking into the NC desktop client git repoository grouped the numbers of commits of people that can be associated clearly to either the ownCloud- or Nextcloud project, or to “other communities” or machine commits. Since the number of commits could be misleading (maybe some commits are huge?) I did the same exercise with numbers of changed lines of code.

When looking on the changed lines, the first top six contributors to the Nextcloud desktop client are only active in the ownCloud project. Number seven is an “other community” contributor whos project the client was based on in the beginning. Number eight to eleven go to Nextcloud, with a low percentage figure.

[caption id=“attachment_944” align=“alignnone” width=“666”] # of commits to the Nextcloud Desktop repository as of Nov. 10, 2018[/caption]

As a result, far more than 80% of the changed lines of the Nextcloud client is actually work that ownClouders did (not considering the machine commits). In the past, and also today. The number would be even higher if it considered all the commits that go into the NC repo with an NC author, but are actually ownCloud patches where the original author got lost on the way by merging them through a NC branch. It looks like the Nextcloud developers were actually adding less commits to their client than all “other community” developers so far.

No wonder, it is a fork, you might think, and that is of course true. However, to my taste these numbers are not reflecting a “constructive” fork driving things forward when we talk about sync technology.

That is all fine, and I am proud that the work we do in ownCloud is actually stimulating two projects, with different focus areas nowadays. On the other hand, I would appreciate if the users of the technology would take a closer look to understand who really innovates, drives things forward and also fixes the nasty bugs in the stack. As a matter of fairness, that should be acknowledged. That is the motivation that keeps free software contributors busy and communities proud.

Heidolph is a mature German engineering company building premium laboratory equipment. It is based in Schwabach, Germany. For me it is the first time that I am working in company that doesn’t do only software. At Heidolph, software is just one building block besides mechanical and electronic parts and tons of special know how. That is a very different situation and a lot to learn for me, but in a small, co-located team of great engineers, I am able to catch up fast in this interesting area.

We build software for the next generation Heidolph devices based on Linux and C++/Qt. Both technologies are in the center of my interest, over the years it has become more than clear for me that I want to continue with that and deepen my knowledge even more.

Since the meaning of open source has changed a lot since I started to contribute to free software and KDE in particular, it was a noticeable but not difficult step for me to take and move away from a self-proclaimed open source company towards a company that is using open source technologies as one part of their tooling and is interested in learning about the processes we do in open source to build great products. An exciting move for me where I will learn a lot but also benefit from my experience. This of course that does not mean that I will stop to contribute to open source projects.

We are still building up the team and look for a Software Quality Engineer. If you are interested in working with us in an exciting environment, you might wanna get in touch.

A couple of days ago version 0.82 was released. It mainly is a bugfix release, but it also comes with a few new features. Users were asking for some new functions that they needed to switch to Kraft with their business communication, and I am always trying to make that a priority.

The most visible feature is a light rework of the calculation dialog that allows users to do price calculations for templates. It was cleared up, superflous elements were finally removed and the remaining ones now work as expected. The distinction between manual price and calculated price should be even more clear now. Time calculations can now not only done in the granularity of minutes, as this was to coarse for certain usecases. The unit for a time slice can now be either seconds, minutes or hours.

[caption id=“attachment_936” width=“516”] New calculation dialog in 0.82[/caption]

Apart from that, for example sending documents per email was fixed, and in addition to doing it through thunderbird, Kraft can now also utilize the xdg-email tool to work with the desktop standard mail client, such as KMail.

Quite a few more bugfixes make this a nice release. Check the full Changelog! Update is recommended.

Thanks for your comments or suggestions about Kraft!

Version 0.81 is a bugfix release for the previous version 0.80, which was the first stable release based on Qt5 and KDE Frameworks5. Even though it came with way more new features than just the port, it’s first release has proven it’s stability in day-to-day business now for a few month.

Kraft 0.81 mainly fixes building with Qt 5.11, and a few other installation- and AppStream metadata glitches. The only user visible fix is that documents do not show the block about individual taxes on the PDF documents any more if the document only uses one tax rate.

Thanks for your suggestions and opinions that you might have about Kraft!

Kraft is desktop software to manage documents like quotes and invoices in the small business. It focuses on ease of use through an intuitive GUI, a well choosen feature set and ensures privacy by keeping data local.

After more than a dozen years of life time, Kraft is now reaching a new level: It is now completely ported to Qt5 / KDE Frameworks 5 and with that, it is compatible with all modern Linux distributions again.

KDE Frameworks 5 and Qt5 are the best base for modern desktop software and Kraft integrates seamlessly into all Linux desktops. Kraft makes use of the great KDE PIM infrastructure with KAddressbook and Akonadi.

In addition to the port that lasted unexpectedly over 12 months, Kraft v. 0.80 got a whole bunch of improvements, just to name some examples:

More Flexible Addressbook Integration

As Akonadi is optional now, Kraft can be built without it. Even if it was built with, but Akonadi for whatever reason is not working properly, Kraft still runs smoothly. In that case it only lacks the convenience of address book integration.

The Address book access was also nicely abstracted so that other Addressbook backends can be implemented more easily.

GUI Improvements

Even though the functionality and GUI of Kraft was not changed dramatically compared to the last stable KDE 4 version, there were a few interesting changes in the user interface.

• A new, bigger side bar simplifies navigation.
• In the timeline view, a click on years and month in the treeview show summaries of the selected time span, ie. the number of documents with financial summaries per month or year.
• A filter allows to limit the view on the current week or month.

Reduction of dependencies

Kraft makes broad use of the core Qt5 libraries. The required KDE dependencies were reduced to a bare minimum. Akonadi libraries, which enable KDE PIM integration are now optional. The former dependency on heavyweight web browser components were completely removed and replaced by the far more simple richtext component of Qt.

These changes make it not only easier and more transparent to build Kraft but allow make a port to other platforms like MacOSX more easy in the future.

Under the Hood

A countless number of bugfixes and small improvements went in. Also updates to the newer C++ concepts where applicable make the rather mature code base more modern and better maintainable.

The Reportlab based PDF document creation script was updated and merged with a later version for example.

Deployment

Installing Kraft is still a bit complicated for unexperienced users, and distributions sometimes haven’t made a good job in the past to provide the latest version of Kraft.

To make it easier to test, there is an AppImage of Kraft 0.80 available that should be runable on most modern distributions. Just download a single file that can be started right away after having added the executable permissions.

Linux packages are already built for openSUSE (various versions) or Gentoo.

Kraft’s website will contain a lot more information.

There was one reaction however, that puzzles me a bit and I want to share my thoughts here. It is about a comment about my announcement that I prefer to continue to develop Kraft on Github. The commenter reminded my friendly that there is still Kraft code on KDE infrastructure, and that switching to a different repository might waste peoples time when they work with the KDE repo.

That is a fair statement, of course I don’t want to waste peoples time. What sounds a bit strange to me is the second paragraph, that says that if I decide to stay with Github, I should let KDE people know that I wish Kraft to not be a KDE project anymore.

But … I never felt that Kraft should not be a KDE project any more.

A little History

Kraft has come a long way together with KDE. I started Kraft in (probably) 2004, gave a talk about Kraft at the Akademy Dublin 2006, maintained it with the best effort I could contribute until today. There is a small but loyal community around Kraft.

During all the time I got little substancial contribution to the code directly, with the exception of one cool developer who got interested for some time and made some very interesting contributions.

When I asked a for the subdomain http://kraft.kde.org long time ago I got the reply that it is not in the interest of KDE to give every little project a subdomain. As a result I reserved http://volle-kraft-voraus.de and run it since then, happily showing a “Part of the KDE family” logo on it.

Beside the indirect contributions to libraries that Kraft uses, I shipped Kraft with the translations made by the KDE i18n team, for which I always was very grateful. Otherwise I got no other services from KDE.

Why Github?

Githubs workflow serves me well in my day job, and since I have only little time for Kraft, I like to use the tools that I know best and give me the most efficiency.

I know that Github is not free software and I am sceptical about that. But Github also does not lock in, as we still are on git. We all know the arguments that usually come on the table at this point, so I am not elaborating here. One thing I want to mention though is that since I moved to Github publically I already got two little pull requests with code contributions. That is a lot compared to what came in the last twelfe years when living on KDE infrastructure only.

Summary

Kraft is a small project, driven by me alone. My development turnaround is good with Github as I am used to it. Even if no KDE developer would ever look at Github (which I know is not true) I have to say with heavy heart that Kraft would not take big harm by leaving KDEs infra, based on the experience of the last 12 years.

If the KDE translation teams do not want to work with Github, I am fine to accept that, and wonder if there could be a solution rather than switching to Transifex.

One point however I like to make very clear: I did not wish to leave KDE, nor aimed to move Kraft out. I still have friends in the KDE community, I am still very interested in free software on desktop and elsewhere, and my opinion is still that KDE is the best around.

If the KDE community feels that Kraft must not be a KDE project any longer because it is on Github, ok. I asked KDE Sysadmins to remove Kraft from the KDE git, and it is already done.

Kraft now lifes on on Github.

Kraft is around for more than twelve years, but it has been a little quiet recently. However, Kraft is alive and kicking!

I am very happy to announce the first public beta version of Kraft V. 0.80, the first Kraft version that is based on KDE Frameworks 5 and Qt 5.x.

It did not only go through the process of being ported to Qt5/KF5, but I also took the opportunity to refactor and tackle a lot of issues that Kraft was suffering from in the past.

Here are a few examples, a full changelog will follow:

• Akonadi dependency: Earlier Kraft versions had a hard dependency on Akonadi, because it uses the KDE Addressbook to manage customer addresses. Without having Akonadi up and running, Kraft was not functional. People who were testing Kraft without having Akonadi up were walking away with a bad impression of Kraft.

  Because Akonadi and the KDE contacts integration is perfect for this use case, it still the way to go for Kraft, and I am delighted to build on such strong shoulders. But Kraft now also works without Akonadi. Users get a warning, that the full address book integration is not available, but can enter addresses manually and continue to create documents with Kraft. It remains fully functional.

  Also, a better abstraction of the Akonadi-based functionality in Kraft eases porting to platforms where other system address books are available, such as MacOSX.

• AppImage: The new Kraft is available as AppImage.

  There was a lot of feedback that people could not test Kraft, because it was hard to set up or compile, and dependency are missing. The major Linux distributions seem to be unable to keep up with current versions of leaf packages like Kraft, and I can not do that for the huge variety of distros. So AppImage as container format for GUI applications seems to be a perfect fit here.

• A lot more was done. Kraft got simplifications in both the code base and the functionality, careful gui changes, and a decreased dependency stack. You should check it out!

Today (on FOSDEM weekend, which could be a better date?) the pre version 0.80 beta10 is announced to the public.

I would appreciate if people test and report bugs at Github: That is where the development is currently happening.

[caption id=“teakear1” align=“alignnone” width=“800”] TeakEar media player[/caption]

This is a Raspberry Pi based audiophile net player that decodes my mp3 collection and net radio to my Linn amplifier. It is called TeakEar, because it’s main corpus is made from teak wood. Obviously I do not want to waste rain forest trees just because of my funny ideas, the teak wood used here has been a table from the 1970ies, back when nobody cared about rainforests. I had the chance to safe parts of the table when it was sorted out, and now use it’s valuable wood for special things.

Upcycling is the idea here, and the central iron part has been a drawbar part from an old hay cart. The black central stone piece is schist, which I found outside in the fields, probably a roofer material.

[caption id=“attachment_902” align=“alignright” width=“220”] Backside view[/caption]

I wanted to combine these interesting and unique materials to something that stands for it’s own. And even though it covers a high tech sound device, I decided against any displays, lights or switches to not taint the warm and natural vibes of the heavy, dark and structured exotic wood, the rusty iron part, which obviously is the hand work of a blacksmith many years ago, and the black stone. Placed on my old school Linn Classic amplifier, it is an interesting object in my living room.

On it’s backside it has a steel sheet fitted into the wood body, held in place by two magnets to allow it to be quickly detached. On that, a Raspberry Pi 2 with a Hifiberry DAC is mounted. The Hifiberry DAC has a chinch output which is connected to the AUX input of the Linn Classic Amplifier. The energy supply is an ordinary external plug in power supply, something that could be improved.

On the software side, I use Volumio, the open audiophile music player. The Volumio project could not amaze me more. After a big redesign and -implementation last year, it is back with higher quality than before but still the same clear focus on music playing. It does nothing else, but with all features and extension points that are useful and needed.

[gallery ids=“921,920,922,923,924” type=“rectangular”]

I enjoy the hours in my workshop building these kind of devices, and I am sure this wont be the last one. Maybe next time with a bit more visible tech stuff.

The conference program is impressive and if you can make it, you should consider stopping by.

Stefan Schäfer from the Invis server project and me will organize a workshop about openSUSE for Small and Medium Business (SMB).

SMB is a long running concern of the heart of the two of us: Both Stefan, who even does it for living, and me have both used openSUSE in the area of SMB for long and we know how well it serves there. Stefan has even initiated the Invis Server Project, which is completely free software and builds on top of the openSUSE distributions. The Invis Server adds a whole bunch of extra functionality to openSUSE that is extremely useful in the special SMB usecase. It came a long way starting as Stefans own project long years ago, evolving as proper maintained openSUSE Spin in OBS with a small, but active community.

The interesting question is how openSUSE, Invis Server and other smaller projects like for example Kraft can unite and offer a reliable maintained and comprehensive solution for this huge group of potential users, that is now locked in to proprietary technologies mainly while FOSS can really make a difference here.

In the workshop we first will introduce the existing projects briefly, maybe discuss some technical questions like integration of new packages in the openSUSE distributions and such, and also touch organizational question like how we want to setup and market openSUSE SMB.

Participants in the workshop should not expect too much presentation. We rather hope for a lively discussion with many people bringing in their projects that might fit, their experiences and ideas. Don’t be shy :-)

Recently an ownCloud fork was coming up with a little shiny box with one harddisk, that can be complemented with a Rapsberry Pi and their software, promoting that as your private cloud.

While I like the idea of building a private cloud for everybody (I started to work on ownCloud because of that idea back in the days), I do not think that this example of gear is a good solution for private cloud.

In fact I believe that throwing this kind of implementations on the table is especially unfortunate because if we come up with too many not optimal proposals, we waste the  willingness of users to try it. This idea should not target geeks who might be willing to try ideas on and on. The idea of the private cloud needs to target at every computer user who wants to store data safely, but does not want to care about longer than ever necessary. And with them I fear we only have very little chances, if one at all, to introduce them to a private cloud solution before they go back to something that simply works.

Here are some points why I think solutions like the proposed one are not good enough:

Hardware

That is nothing new: The hardware of the Raspberry Pi was not designed for this kind of usecases. It is simply too weak to drive ownCloud, which is an PHP app plus database server that has some requirements on the servers power. Even with PHP7, which is faster, and the latest revisions of the mini computer, it might look ok in the beginning, but after all the neccessary bells and whistles were added to the installation and data run in, it will turn out that the CPU power is simply not enough. Similar weaknesses are also true for the networking capabilities for example.

A user that finds that out after a couple of weeks after she worked with the system will remain angry and probably go (back) to solutions that we do not fancy.

One Disk Setup

The solution comes as one disk setup: How secure can data be that is on one single hardisk? A seriously engineered solution should at least recommend a way to store the data more securely and/or backup, like on an at homes NAS for example. That can be done, but requires manual work and might require more network capabilities and CPU power.

Advanced Networking

Last, but for me the most important point: Having such a box in the private network requires to drill a whole in the firewall, to allow port forwarding. I know, that is nothing unusual for experienced people, and in theory little problem.

But for people who are not so interested, that means they need to click in the interface of their router on a button that they do not understand what it does, and maybe even insert data by following an documentation that they have to believe. (That is not very much different from downloading a script from somewhere letting it do the changes which I would not recommend as well). Doing mistakes here could potentially have a huge impact for the network behind the router, without that the person who did it even has an understanding for.

Also DynDNS is needed: That is also not a big problem in theory and for geeks, but in practice it is nothing easily done.

With a good solution for private cloud, it should not be necessary to ask for that kind of setups.

Where to go from here?

There should be better ways to solve this problems with ownCloud, and I am sure ownCloud is the right tool to solve that problem. I will share some thought experiments that we were doing some time back to foster discussion on how we can use the Raspberry Pi with ownCloud (because it is a very attractive piece of hardware) and solve the problems.

This will be subject of an upcoming blog here, please stay tuned.

There has been the big enterprise release 9.1 on September 20th, but that of course came along with community releases which are in the focus here.

We had server release 8.0.15, server release 8.1.10, server release 8.2.8 and release 9.0.5. There are maintenance releases for the older major versions, needed to fix bugs on installations that still run on these older versions. We deliver them following this plan.

The latest and greatest server release is release 9.1.1 that has all the hardening that also went into the enterprise releases.

Aside a ton of bugfixes that you find listed in the changelog there have also been interesting changes which drive innovation. To pick just one example: The data fingerprint property. It enables the clients to detect if the server got a backup restored, and saves changes on the clients to conflict files if needed. This is a nice example of solutions which are based on feedback from enterprise customers community running ownCloud, who help with reporting problems and proposing solutions.

Talking about professional usage of ownCloud: Of course also all the server release are available as linux packages for various distributions, for example the ownCloud server 9.1.1 packages. We think that our users should not be forced to deploy from tarballs, which is error prone and not native to Linux, but have the choice to use linux packages through the distributions package management.

There also have been client releases recently: The Android client versions 2.1.1 and 2.1.2 were released with important changes for Android 7 and much more fixes, as well as iOS client versions 3.5.0 and 3.5.1. The desktop client 2.2.4 also got a regular bug fix update (Changelog).

I guess you agree that is a lot of activity shown in the ownCloud project, making sure to get the best ownCloud experience out there for the users, driven by passion for the project and professional usage in focus.

If you are interested and want to join in and make ownCloud better, jump in on ownCloud Central or Github. It’s fun!

But there is even more opportunity: Additionally we are looking for an ownCloud Desktop Client developer. That would be somebody fluid in C++ and Qt who likes to pick up responsibility for our desktop client together with the other guys on the team. Shifted responsibilities have created this space, and it is your chance to jump into the desktop sync topic which makes ownCloud really unique.

The role includes working with the team to plan and roll out releases, coordinate with the server- and mobile client colleagues, nail out future developments, engage with community hackers and help with difficult support cases. And last but not least there is hacking fun on remarkable nice Qt based C++ code of our desktop client, together with high profile C++ hackers to learn from.

It is an ideal opportunity for a carer type of personality, to whom it is not enough to sit in the basement and only hack, but also to talk to people, organize, and become visible. Having a Qt- and/or KDE background is a great benefit. You would work from where you feel comfortable with as ownCloud is a distributed company.

The ownCloud Client is a very successful part of the ownCloud platform, it has millions of installations out there, and is released under GPL.

If you want to do something that matters, here you are! Send your CV today and do not forget to mention your github account :-)

What we do - what you will work on

The call is for people who understand the vision of bringing the idea ownCloud to an enterprise ready level: ownCloud is not only running on individual open source enthusiasts hardware, but also on sites with huge amounts of data like CERN or the Sciebo project, and at large companies who want to work with their data in a secure way.

To provide the best solution for all of them we are looking for:

A System Administrator

In this role, you make sure that the infrastructure that we use in ownCloud is up and running. That involves troubleshooting and streamlining existing infrastructure, but also designing new services. If you love virtualization of all kinds and have an eye for security, this position is for you. Of course all this does not only happen behind closed doors, but you will be in contact with the open source community around ownCloud.

The Application Security Engineer

For security professionals who would like to take on a high profile open source project. As security is one of the core values of ownCloud, we are looking for somebody who constantly monitors the code flowing in for security problems, is able to find glitches in existing code and handle the bug bounty program. That and more is the task of this high profile position.

A Software Engineer PHP

For engineers with a passion for good software design and a love for writing code without being code monkeys: In this role you iron the server part of our platform, build new features, work on fixing bugs with the support colleagues and bother the architect with new ideas how to make the thing even better. For this you need to urge to get down and dirty with code, feel yourself comfortable in a team of high profile developers who can teach you things and learn from you.

PHP or what?

Yes, ownCloud is written in PHP, and PHP is the most important, but by far not the only language that we use for the ownCloud platform.

Before you turn your back because of PHP, please think twice. There are a lot of good reasons why we are going with PHP, some of them are named in this blog, but there is more: For example PHP7: With PHP 7 (which can be used with ownCloud) the language has caught up with many criticism it faced before and has done a big leap.

And anyway, the language of a system is not the only thing that is important in a developers life. It is rather how many people use, love and recommend the project and the development processes the team lives. And in all that points, ownCloud is already awesome, and will become even more with your help.

Send your resume in to work@owncloud.com so we can get talking!

Overlay Icons

Overlay icons for the various file managers on our three platforms already exist for quite some time, but it has turned out that the performance was not up to the mark for big sync folders. The reason was mainly that too much communication between the file manager plugin and the client was happening. Once asked about the sync state of a single file, the client had to jump through quite some hoops in order to retrieve the required information. That involved not only database access to the sqlite-based sync journal, but also file system interaction to gather file information. Not a big deal if it’s only a few, but if the user syncs huge amounts, these efforts do sum up.

This becomes especially tricky for the propagation of changes upwards the file tree. Imagine there is a sync error happening in the foo/bar/baz/myfile. What should happen is that a warning icon appears on the icon for foo in the file manager, telling that within this directory, a problem exists. The complexity of the existing implementation was already high and adding this extra functionality would have reduced the reliability of the code lower than it already was.

Jocelyn was keen enough to do a refactoring of the underlying code which we call the SocketApi. Starting from the basic assumption that all files are in sync, and the code has just to care for these files that are new or changed, erroneous or ignored or similar, the amount of data to keep is very much reduced, which makes processing way faster.

Server Notifications

On the ownCloud server, there are situation where notifications are created which make the user aware of things that happened.

An example are federated shares:

If somebody shares a folder with you, you previously had to acknowledge it through the web interface. This explicit step is a safety net to avoid people sharing tons of Gigabytes of content, filling up your disk.

With 2.2.x, you can acknowledge the share right from the client, saving you the round trip to the web interface to check for new shares.

Keeping an Eye on Word & Friends

Microsoft Word and other office tools are rather hard to deal with in syncing, because they do very strict file locking of the files that are worked on. So strict that the subsequent sync app is not even allowed to open the file, not even for reading. That would be required to be able to sync the file.

As a result the sync client needs to wait until word unlocks the file, and then continue syncing.

For previous version of the client, this was hard to detect and worked only if other changes happened in the same directory where the file in question resides.

With 2.2.0 we added a special watcher that keeps an eye on the office docs Word and friends are blocking. And once the files are unlocked, the watcher starts a sync run to get the files to the server, or down from the server.

Advances on Desktop Sharing

The sharing has been further integrated and received several UX- and bugfixes. There is more feedback when performing actions so you know when your client is waiting for a response from the server. The client now also respect more data returned from the server if you have apps enabled on the server that for example limit the expiration date.

Further more we better respect the share permissions granted. This means that if somebody shared a folder without create permissions with you and you want to reshare this folder in the client you won’t get the option to share with delete permissions. This avoids errors when sharing and is more in line with how the whole ownCloud platform handles re-sharing. We also adjusted the behavior for federated reshares with the server.

Please note to take full advantage of all improvements you will need to run at least server version 9.0.

Have fun!

But not Volumio.

After quite some development time the project released RC1. While there were a few small bugs in a beta, my feelings about the RC1 are really positive. Volumio2 has a very nice and stylish GUI, a great improvement over Volumio1. Album-art is now nicely integrated in the playback pane and and everything is more shiny, even if the general concept is the same as in Volumio1.

I like it because it is only a music player. Very reduced on that, but also very thought through and focussed to fulfill that job perfectly. I just want to find and play music from my collection, quickly and comfortable and with good sound quality. No movies, series, images. Just sound.

About speed: While the scanning of my not too big music collection on a NAS was a bit of a time consuming task in the past, this feels now much faster (maybe thats only because of a faster network between the Raspberry and the NAS?). Searching, browsing and everything works quite fluid on an Raspberry2. And with the Hifiberry DAC for output, the sound quality is more than ok.

This is an release candidate of the first release of the rewritten project, and the quality is already very good. Nevertheless I found a few things that did not work for me or could be improved. That the volume control is not working is probably because of the Hifiberry DAC driver, I remember there was something, but haven’t investigated yet.

There are some things in the GUI that could be looked at again: For example on the Browse page, there is the very well working search field. After entering the search term and Enter, the search result is displayed as a list of songs to select from. I wished that the songs were additionally grouped by albums, which should also be selectable to be pushed to the play queue.

Also it would be great if the Queue would somehow indicate which entry is currently played. I could not spot that.

But these are only minor findings which can easily be addressed later after enhancement requests were posted :-)

I think Volumio2 is already a great success, even before it was released! You should not hesitate to try it if you love to listen to music!

Thanks for the hard work Volumio-Team!

This part makes a couple of ideas how the new chunking could be useful with a future feature of incremental sync (also called delta sync) in ownCloud.

In preparartion of delta sync the server could provide another new WebDAV route: remote.php/dav/blocks.

For each file, remote.php/dav/blocks/file-id exists as long as the server has valid checksums for blocks of the file which is identified by its unique file id.

A successful reply to remote.php/dav/blocks/file-id returns an JSON formatted data block with byte ranges and the respective checksums (and the checksum type) over the data blocks for the file. The client can use that information to calculate the blocks of data that has changed and thus needs to be uploaded.

If a file was changed on the server and as a result the checksums are not longer valid, access to remote.php/blocks/file-id is returning the 404 “not found” return code. The client needs to be able to handle missing checksum information at any time.

The server gets the checksums of file blocks along the upload of the chunks from the client. There is no obligation of the server to calculate the checksums of data blocks that came in other than through the clients, yet it can if there is capacity.

To implement incremental sync, the following high level processing could be implemented:

1. The client downloads the blocklist of the file: GET remote.php/dav/blocks/file-id
2. If GET succeeded: Client computes the local blocklist and computes changes
3. If GET failed: All blocks of the file have to be uploaded.
4. Client sends request MKCOL /uploads/transfer-id as described in an earlier part of the blog.
5. For blocks that have changed: PUT data to /uploads/transfer-id/part-no
6. For blocks that have NOT changed: COPY /blocks/file-id/block-no /uploads/transfer-id/part-no
7. If all blocks are handled by either being uploaded or copied: Client sends MOVE /uploads/transfer-id /path/to/target-file to finalize the upload.

This would be an extension to the previously described upload of complete files. The PROPFIND semantic on /uploads/transfer-id remains valid.

Depending on the amount of not changed blocks, this could be a dramatic cut for the data that have to be uploaded. More information has to be collected to find out how much that is.

Note that this is still in the idea- and to-be-discussed state, and not yet an agreed specification for a new chunking algorithm.

Please, as usual, share your feedback with us!

Hardware

The device is based on a Raspberry Model B+ with a HifiBerry Amp+. [caption id=“attachment_618” align=“alignleft” width=“446”] Cherry Ear backside with connectors and power supply[/caption]The Amp+ is a high-quality power amplifier that is mounted on the Raspberry mini computer, shortcutting the sub optimal audio system of the raspy. Only loudspeakers need to be connected, and this little combination is a very capable stereo audio system.

Software: Volumio

On the Raspberry runs a specialised linux distribution with a web based audio player for audio files and web radio. The name of this project is Volumio and it is one of my most favorite projects currently. What I like with it is that it is only a music player, and not a project that tries to manage all kind of media. Volumios user interface is accessible via web browser, and it is very user friendly, modern, pretty and clean.

It works on all size factors (from mobile phone to desktop) and is very easy to handle. Its a web interface to mpd which runs on the raspi for the actual music indexing and playing.

[caption id=“attachment_619” align=“alignleft” width=“400”] Volumio User Interface[/caption]The distribution all is running on is an optimized raspbian, with a variety of kernel drivers for audio output devices. Everything is pre configured. Once the volumio image is written to SD-Card, the Raspberry boots up and the device starts to play nicely basically after network and media source was configured through the web interface.

It is impressive how the Volumio project aims for absolute simplicity in use, but also allows to play around with a whole lot of interesting settings. A lot can be, but very little has to be configured.

Bottomline: Volumio is really a interesting project which you wanna try if you’re interested in these things.

The Housing

I built the housing out of cherry tree from the region here. I got it from friends growing cherries, and they gave a few very nice shelfs. It was sliced and planed to 10mm thickness. The dark inlay is teak which I got from 70’s furniture that was found on bulky waste years ago.

After having everything cut, the cherry wood was glued together, with some internal help construction from plywood. After the sanding and polishing, the box was done.

[caption id=“attachment_621” align=“alignright” width=“300”] Sizes [cm][/caption]The Raspberry and Hifiberry are mounted on a little construction attached to the metal back cover, together with the speaker connector. The metal cover is tightend with one screw, and the whole electronics can be taken off the wood box by unscrewing it.

At the bottom of the device there is a switching power supply that provides the energy for the player.

How to Operate?

The player is comletely operated from the web interface. The make all additional knobs and stuff superflous, the user even uses the tablet or notebook to adjust the volume. And since the device is never switched off, it does not even have a power button.

I combined it with Denon SC-M39 speakers. The music files come from a consumer NAS in the home network. The Raspberry is very well powerful enough for the software, and the Hifiberry device is surprisingly powerful and clean. The sound is very nice. Clear and fresh in the mid and heights, but still enough base, which never is annoying blurry or drones.

I am very happy about the result of this little project. Hope you like it :-)

With the processing described in the first part of the blog, the upload is done savely and with a clean approach, but it also has some drawbacks.

Most notably the server does not know the target filename of the uploaded file upfront. Also it does not know the final size or mimetype of the target file. That is not a problem in general, but imagine the following situation: A big file should be uploaded, which would exceed the users quota. That would only become an error for the user once all uploads happened, and the final upload directory is going to be moved on the final file name.

To avoid useless file transfers like that or to implement features like a file firewall, it would be good if the server would know these data at start of the upload and stop the upload in case it can not be accepted.

To achieve that, the client creates a file called _meta in /uploads/ before the upload of the chunks starts. The file contains information such as overall size, target file name and other meta information.

The server’s reply to the PUT of the _meta file can be a fail result code and error description to indicate that the upload will not be accepted due to certain server conditions. The client should check the result codes in order to avoid not necessary upload of data volume of which the final MOVE would fail anyway.

This is just a collection of ideas for an improved big file chunking protocol, nothing is decided yet. But now is the time to discuss. We’re looking forward to hearing your input.

The third and last part will describe how this plays into delta sync, which is especially interesting for big files, which are usually chunked.

This is the first of three little blog posts in which we want to present the idea and get your feedback. This is for open discussion, nothing is set in stone so far.

What is the downside of the current approach? Well, the current algorithm needs a lot of distributed knowledge between server and client to work: The naming scheme of the part files, semi secret headers, implicit knowledge. In addition to that, due to the character of the algorithm the server code is too much spread over the whole code base which makes maintaining difficult.

This situation could be improved with the following approach.

To handle chunked uploads, there will be a new WebDAV route, called remote.php/uploads. All uploads of files larger than the chunk size will go through this route.

In a nutshell, an upload of a big file will happen as parts to a directory under that new route. The client creates it through the new route. This initiates a new upload. If the directory could be created successfully, the client starts to upload chunks of the original file into that directory. The sequence of the chunks is set by the names of the chunk files created in the directory. Once all chunks are uploaded, the client submits a MOVE request the renames the chunk upload directory to the target file.

Here is a pseudo code description of the sequence:

1. Client creates an upload directory with a self choosen name (ideally a numeric upload id):

MKCOL remote.php/uploads/upload-id

2. Client sends a chunk:

PUT remote.php/uploads/upload-id/chunk-id

3. Client repeats 2. until all chunks have successfully been uploaded 4. Client finalizes the upload:

MOVE remote.php/uploads/upload-id /path/to/target-file

5. The MOVE sends the ETag that is supposed to be overwritten in the request header to server. Server returns new ETag and FileID as reply headers of the MOVE.

During the upload, client can retrieve the current state of the upload by a PROPFIND request on the upload directory. The result will be a listing of all chunks that are already available on the server with metadata such as mtime, checksum and size.

If the server decides to remove an upload, ie. because it hasn’t been active for a time, it is free to remove the entire upload directory and return status 404 if a client tries to upload to. Also, a client is allowed to remove the entire upload directory to cancel an upload.

An upload is finalized by the MOVE request. Note that it’s a MOVE of a directory on a single file. This operation is not supported in normal file systems, but we think in this case, it has a nice well descriptive meaning. A MOVE is known as an atomic and fast operation, and that way it should be implemented by the server.

Also note that only with the final MOVE the upload operation is associated with the final destination file. We think that this approach already is a great improvement, because there is always a clear state of the upload with no secret knowledge hidden in the process.

In the next blog I will discuss an extension to this that adds more features to the process.

What do you think so far? Your feedback is appreciated, best on the ownCloud devel mailinglist!

[caption id=“attachment_586” align=“alignleft” width=“300”] The Share Dialog[/caption]This release brings a new integration into the operating system file manager. With 1.8.0, there is a new context menu that opens a dialog to allow the user to create a public link on a synced file. This link can be forwarded to other users who get access to the file via ownCloud.

Also the clients behavior when syncing files that are opened by other applications on Windows has greatly been improved. The problems with file locking some users saw for example with MS office apps were fixed.

Another area of improvements is again performance. With latest ownCloud servers, the client uses even more parallized requests, now for all kind of operations. Depending on the synced data structure, this can make a huge difference.

All the other changes, improvements and bug-fixes are too hard to count. Finally, this release received around 700 git commits compared to the previous release.

All this is only possible with the powerful and awesome community of ownClouders. We received a lot of very good contributions through the GitHub tracker, which helped us to nail down a lot of issues and improved the client tremendously.

But this time we’d like to specifically point out the code contributions of Alfie “Azelphur” Day and Roeland Jago Douma who contributed significant code bits to the sharing dialog on the client and also some server code.

A great thanks goes out to all of you who helped with this release. It was a great experience again and it is big fun working with you!

We hope you enjoy 1.8.0! Get it from https://owncloud.org/install/#desktop

ETags

ETags are strings that describe exactly one specific version of a file (example: 71a89a94b0846d53c17905a940b1581e).

Whenever the file changes, the ownCloud server will make sure that the ETag of the specific file changes as well. It is not important in which way the ETag changes, it also does not have to be strictly unique, it’s just important that it changes reliably if the file changes for whatever reason. However, ETags should not change if the file was not changed, otherwise the client will download that file again.

In addition to that, The ETags of the parent directories of the file have to change as well, up to the root directory. That way client systems can detect changes that happen somewhere in the file tree. This is in contrast to normal computer file systems where only the modification time of the direct parent of a file is changing.

File IDs

FileIDs are also strings that are created once at the creation time of the file (example: 00003867ocobzus5kn6s).

But contrary to the ETags, the file IDs should never ever change over the files lifetime. Not on an edit of the file, and also not if the file is renamed or moved. One of the important usages of the FileID is to detect renames and moves of a file on the server.

The FileID is used as an unique key to identify a file. FileIDs need to be unique within one ownCloud, and in inter-owncloud connections, they must be compared together with the ownCloud server instance id.

Also, the FileIDs must never be recycled or reused.

Checksums?

Often ETags and FileIDs are confused with checksums such as MD5 or SHA1 sums over the file content.

Neither ETags nor FileIDs are, even if there are similarities: Especially the ETag can be seen as a checksum over the file content. However, file checksums are way more costly to compute than just a value that only needs to change somehow.

What happens if…?

Let’s make a thought experiment and consider what it would mean especially for sync clients if either fileID or ETag gets lost from the servers database.

If ETags are lost, clients loose the ability to decide if files have changed since the last time that was checked by the clients. So what happens is that the client will download the files again, byte-wise compare them to the local file and use the server file if the files differ. A conflict file will be created. Because the ETag was lost, the server will create new ETags on download. This could be improved by the server creating more predictable ETags based on the storage backends capabilities.

If the ETags are changed without reason, for example because a backup was played back on the server, the clients will consider the ones with changed ETags as changed and redownload them. Conflict handling will happen as described if there was a local change as well.

For the user, this means a lot of unnecessary downloads as well as potential conflicts. However, there will not be data loss.

If FileIDs got lost or changed, the problem is that renames or moves on server side can no longer be detected. That would result in a new download of files in the good case. If a fileID however changes to something that was used before, that can result in a rename that overwrites an unrelated file. That is because clients might still have the FileID associated with another file.

Hopefully this little post explains the importance of the additional metadata that we maintain in ownCloud.

RSync is great

Nobody will object here. In a nutshell, this is how rsync works: There is a file on the client and on the server. The idea is to not transfer the entire file from one side to the other if either side changes, but only the parts that have changed.

The original rsync does that by chopping the file to blocks of a given size and calculating a checksum of each of the blocks. The list of checksums is sent to the server and - here’s the trick - the server looks at its version of the file and for each of the checksum in the list, it seeks if it finds the same block in the file. That will often not be at the same position in the file, but maybe somewhere else. That is done for each block, and finally the server will work out the information of which parts of the file are existing and which are not and have to be sent by the client.

By way of this clever algorithm, we will just have to transmit a very small fraction of the changed file, because most content did not change. And that is what we want! Yeah!

Mission accomplished? No, not really. While there is basically nothing wrong with the idea in general, there is a severe architectural downside. The rsync algorithm depends on a strong server component which, for each file, searches around and calculates checksums. In an environment where we potentially have a lot of clients connecting to one server that would create a huge load on it which we need to avoid. So what if instead of putting the burden on the server’s shoulder, we could make the clients take the responsibility?

And guess what, there has been somebody thinking about that before and he says:

Use ZSync for this!

ZSync basically turns the idea of rsync upside down and shifts the calculation of checksums away from the server and onto the clients. That means that with zsync, the server can keep a static list of checksums for every block specific to a version of a file. The list can for example be computed along the upload of the file to the server. From that point it does not change, as long as the file does not change. That means less computation work for the server, and maybe this job can also put into the client.

So far that sounds cool (even though some questions remain) and sounds like something that can help us.

Unfortunately, the approach does not work very well for compressed files. The reason is that if a file gets compressed, even if only a couple of bytes in the original file change, the compression algorithm usually changes a lot all over the entire file. As a result, the zsync algorithm can only compute a comparably large diff. Given the cost of computation that can turn inefficient quickly.

“But who uses compressed files?” you might argue. The problem is that almost each and every of the files in everyday life are stored compressed. This is for example true for Microsoft Office files and the Open Document files produced by LibreOffice and Apache OpenOffice. They are really renamed ZIP containers, that hold the documents with all its embedded files, etc.

Now of course you will reply that zsync has an improved algorithm for compressed files. Yes, true, that is a great thing. However, it involves that the compressed file gets uncompressed to be worked on by zsync. Afterwards it is compressed again. And that is the problem: As common compressors do not leave a hint behind _how_ the file was compressed, it is not possible to reliably recreate a file that is equivalent to the original one. How will apps react on a file that has changed its compression scheme?

Results

As said above, yes, we will at one point of time implement something along the zsync algorithm. The explanations above should show however, that at the current state of ownCloud, other features will improve ownClouds performance, stability and convenience more. And that is the important thing for us, more than pleasing the loudest barking dogs. Here is a rough outline of how I would move on on this, open for your suggestions and critique: The zsync algorithm is designed to improve downloads. We need it for both up- and downloads, and it needs to be thought through if that is also possible. For the server side functionality, there are a couple of open questions which carefully have to be investigated. Preferably an app can be written that provides the handling of the zsync checksum lists. That has to be clarified and discussed, and that will take a while. But as outlined above, this idea is only clever for a limited amount of file types. So what I would suggest first is that we get an idea of the file types users usually store in their ownCloud, so that we can do a validated estimate on how this feature helps. I will follow up on this first step.

Thanks for reading this long blog post. Thanks Danimo for lectorate.

We prepared some test packages in our development repository isv:ownCloud:devel for those who wanna try and know their way around. Current it only builds for a couple of openSUSE Distros. You need to install kdebase4 and dolphin-plugins and after installation, it’s easiest to restart KDE to make it registered. But be warned: The two packages replace packages from the previous installation, only do it if you really know what you’re doing!

It would be great if at least the libkonq patch could make it to upstream, and I would appreciate if somebody who is a bit more fluent with recent KDE libs development could give me a hand on that. Otherwise, if distros wanna pick up the patches to make the overlays work, of course the patches are here: patch for libkonq and the ownCloud Dolphin plugin. The plugin will work with the released version 1.7.0 of ownCloud Client.

CERN is a very inspiring place, attracting intelligent people from all over the world to get behind the secrets of our being. I felt honored to be at the place where for example the world wide web was invented.

The event was called Workshop on Cloud Services for File Synchronisation and Sharing and was hosted by CERN IT department. There have been around 100 attendees.

I was giving a talk called The File Sync Algorithm of the ownCloud Desktop Clients, which was very well received. If you happen to be interested in the sync algorithm we’re using, the slides are a nice starting point.

What amazed me most was the great atmosphere and the very positive attitude towards ownCloud. Many representatives of edu organizations that use ownCloud to which I talked were very happy with the product (even though there are problems here and there) from the technical POV. A lot of interesting setups and environments were explained and also showcased ownCloud’s flexibility to integrate into existing structures.

What also was pointed out by the attendees of the workshop was the importance of the fact that ownCloud is open source. Non free software does not have a chance at all in that market. That was the very clear statement in the final discussion session of the workshop.

The keynote was given by Prof. Benjamin Pierce from Pennsylvania with the title Principles of Synchronization. He is the lead author of the project Unison which is another opensource sync project. It’s sync engine marks very high quality, but is not “up-to-date software” any more as he said. I had the pleasure to spend quite some time with him to discuss syncing in general and our sync algorithms in particular, amongst other interesting things.

[caption id=“attachment_525” align=“alignleft” width=“254”] Atlas Detectors[/caption]As part of his work, he works with a tool called QuickCheck to do very enhanced testing. One night we were sitting in the cantina there hacking to adopt the testing to ownCloud client and server. The first results were very promising, for example we revealed a “problem” in our sync core that I knew of, which formally is a sync error, yet very very unlikely to happen and thus accepted for the sake of an easier algorithm. It was impressive how fast the testing method was identifying that problem. I like to follow up with the testing method.

Furthermore we met with a whole variety of other interesting people, backend developers, operators of the huge datasets (100 Peta-Byte), the director of CERN IT, a maintainer of the Scientific Linux and others.

Also we had the chance to visit the Atlas experiment, it is 100 meter underneath the surface and huge. That is where the particles are accelerated, and it was great to have the chance to visit that.

The trip was a great experience and very motivating for me, and I think it should be for all of us all doing ownCloud. Frank was really hitting a nerv when he was seeding the idea, and we all were doing a nice product of it so far.

Lets do more of this cool stuff!

The new release brings two lighthouse features which I’ll briefly describe here.

Overlay icons

For the first time, this release has a feature that lives kind of outside the ownCloud desktop client program. That nicely shows that syncing is not only a functionality living in one single app, but a deeply integrated system add-on that affects various levels of desktop computing.

Here we’re talking about overlay icons which are displayed in the popular file managers on the supported desktop platforms. The overlay icons are little additional icons that stick on top of the normal file icons in the file manager, like the little green circles with the checkmark on the screenshot.

The overlays visualize the sync state of each file or directory: The most usual case that a file is in sync between server and client is shown as a green checkmark, all good, that is what you expect. Files in the process of syncing are marked with a blue spinning icon. Files which are excluded from syncing show a yellow exclamation mark icon. And errors are marked by a red sign.

What comes along simple and informative for the user requires quite some magic behind the curtain. I promise to write more about that in another blog post soon.

Selective Sync

Another new thing in 1.7.0 is the selective sync.

In ownCloud client it was always possible to have more than one sync connection. Using that, users do not have to sync their entire server data to one local directory as with many other sync solutions. A more fine granular approach is possible here with ownCloud.

For example, mp3’s from the Music dir on the ownCloud go to the media directory locally. Digital images which are downloaded from the camera to the “photos” dir on the laptop are synced through a second sync connection to the server photo directory. All the other stuff that appears to be on the server is not automatically synced to the laptop which keeps it organized and the laptop harddisk relaxed.

While this is of course still possible we added another level of organization to the syncing. Within existing sync connections now certain directories can be excluded and their data is not synced to the client device. This way big amounts of data can be easier organized depending on the demands of the target device.

To set this up, check out for the button Choose what to Sync on the Account page. It opens the little dialog to deselect directories from the server tree. Note that if you deselect a directory, it is removed locally, but not on the server.

What else?

There is way more we put into this release: A huge amount of bug fixes and detail improvements went in. Fixes for all parts of the application: Performance (such as database access improvements), GUI (such as detail improvements for the progress display), around the overall processing (like how network timeouts are handled) and the distribution of the applications (MacOSX installer and icons), just to name a few examples. Also a lot of effort went into the sync core where many nifty edge cases were analyzed and better handled.

Between version 1.6.2 and the 1.7.0 release more than 850 commits from 15 different authors were pushed into the git repository (1.6.3 and 1.6.4 were continued in the 1.6 branch which commits are also in the 1.7 branch). A big part of these are bug fixes.

Who is it?

Who does all this? Well, there are a couple of brave coders funded by the ownCloud company working on the client. And we do our share, but not everything. Also coding is only one thing. If you for example take some time and read around in the client github repo it becomes clear that there are so many people around who contribute: Reporting bugs, testing again and again, answering silly looking questions, proposing and discussing improvements and all that (yes, and finally coding too). That is really a huge block, honestly.

Even if it sometimes becomes a bit heated, because we can not do everything fast enough, that still is motivating. Because what does that mean? People care! For the idea, for the project, for the stuff we do. How cool is that? Thank you!

Have fun with 1.7.0!

So I tried to investigate the app with Qt Inspector of Robert Knight. That’s a great tool, but it does not yet completely do what I need because it only shows QWidget based objects. But Robert was kind enough to put me on the right track, thanks a lot for that!

I tried this naive approach:

In the clients main.cpp, I implemented these both callback functions:

[code language=“cpp”] QSet<QObject*> mObjects;

extern “C” Q_DECL_EXPORT void qt_addObject(QObject *obj) { mObjects.insert(obj); }

extern “C” Q_DECL_EXPORT void qt_removeObject(QObject *obj) { mObjects.remove(obj); } [/code]

Qt calls these callbacks whenever a QObject is created or deleted respectively. When the object is created I add it’s pointer to the QSet mObjects, and if it is deleted, it is removed from the QSet. My idea was that after the QApp::exec() call returns, I would have to see which QObjects are still in the mObjects QSet. After a longer run of the client, I hoped to see an artificial amount of objects being left over.

Well, what should I say… No success so far: After first tests, it seems that the amount of left-over objects is pretty constant. Also, I don’t see any objects that I would not kind of expect.

So this little experiment left more questions than answer: Is the suspicion correct that QObjects with a valid parent pointer can cause the memory growth? Is my test code as I did it so far able to detect that at all? Is it correct to do the analysis after the app.exec() call returned?

If you have any hints for me, please let me know! How would you tackle the problem?

Thanks!

This is the link to my modified main.cpp: https://github.com/owncloud/mirall/blob/qobject_monitor/src/main.cpp

In my opinion, ownCloud 7 is a real step ahead. So much work went into it from the brave server guys, and the result is simply impressive: It is fast, beautiful and fully focused on what it is supposed to be and do: File sync and share, even though you still have of course all the other useful stuff like calendaring and such.

Apart from the wellknown features ownCloud 7 also brings some things that you do not see on the first look. Some of these things relate to syncing and as a guy working mainly on the sync client I am especially happy about that: Already with the current released clients you will see a performance win in syncing, because ownCloud 7 handles parallel HTTP requests way better. Moreover, ownCloud 7 sends some interesting information about it’s files, and the next generation of sync clients will for example know which files are shared and their permissions. We are currently working on some exciting stuff on the client side, stay tuned.

That all should be reason enough to celebrate together, because ownCloud is done by all of us in the community.

On Tuesday, august 5th, we will do a Release Party in Nuremberg, Germany. As last time, the Coworkingspace Nürnberg is hosting us. We will start at 18:00.

Everybody who is interested in ownCloud for whatever reason is very welcome to show up. We will probably have a short informal demonstration or two, but of course plenty room for questions, discussions, meet and greet with all kind of FOSS people, some core ownCloud people, nice beer and big fun.

You really should come! And don’t forget to tell your friends, thanks!

The recommendation is to update your installation to this version. The previous version 1.6.0 had great new features, first and foremost the parallel up- and download of files and a way more performant handling of the local sync journal. That required a lot of code changes. Unfortunately that also brought in some bugs which are now fixed with the 1.6.1 release.

On the windows platforms, we experienced a memory leak that over time let the memory consumption of the client grow. Also, a problem in the Qt5 library that we ship for windows caused the problem that under some circumstances the wrong encryption lib was loaded, so that some people saw SSL problems on Windows.

And there were crashes. Users kept on reporting that the client was crashing after some time on windows, without a special reason. None of the developers was able to reproduce that or ever saw that. We asked for backtraces, which also can be produced on windows. Even though the backtraces looked similar, we did not find an obvious reason for the crashes. Finally, by reading through all involved code levels again and again, Olivier was able to spot some code in libneon that, again under special circumstances, could cause crashes on win.

It was a one line fix, we quickly built test packages, people tested, and finally the crashes were gone (the patch to libneon is on its way to upstream of course).

All that is now fixed in 1.6.1.

What does that show? There not very much little “easy” bug findings any more. That is similar to the soccer world championship, where the coaches keep telling that there are no “easy opponents” any more nowadays, which is also true. These tricky problems we face in the client are hard to find, require time, often special setups if they are reproduceable at all, and advanced debugging skills. Very challenging, very much fun. But that also requires very much patience from the people who suffer from that bugs. We keep on asking questions, ask to test new daily builds and need time to investigate stuff, and more time to release once we have the fix.

Thank you all for helping in this situation, not giving up, for again testing another daily build, reporting back, trying again. That is really big.

It was finished today and fixes two issues that came up after the release of the previous version: A compile fix for compiling Kraft against the latest version of ctemplate, which did not work out of the box with Kraft 0.54. The other fix is a fail in generating PDF documents which is a pretty severe problem for Kraft. Both is now working well with the new version 0.55 which can be downloaded here.

Thanks a lot to the Kraft community for testing!

If you find other problems, want to give feedback or seek help, please see here how to proceed. Thanks!

It is not only a maintenance release but also comes with a couple of new features, the most outstandig is the ability to handle a new document type, the delivery note which prints no prices. That closes a gap for interesting use cases. Here is a more detailed log of what was added to this release.

Kraft is KDE software to help people driving a small business. Emphasis is on small and business. We are not talking CMS, ERP or any other monster. Kraft is about a handy alternative for people who wrote their first 25 invoices using Libre Office and now start to think of how they could could be more efficient in doing that: Using structured templates, being able to create an invoice based on a quote that was done before, no need to fiddle around with slipping paragraphs, a proper address book, such stuff. Software for people who have other things to do than sit in front of their computer. Hard to understand for geeks like us who enjoy this technology, but yes, there are a lot of people who do not, who just use computers because they must, because they have a business.

I started to work on Kraft in 2006, and worked on similar software before, well, we all have our dark history. I always enjoyed doing software for people who would prefer to not use the computer. And the more I got involved into KDE the more obvious it became to me how perfectly KDE is able to help with that. High level classes, components to reuse, other projects aiming the same direction, and a community of helpful, friendly and open minded people. Also I think software like Kraft is a good addition to the KDE family as it has potential to bring more and different users to KDE.

However, if measured by the number of known users of Kraft, this idea failed completely. Compared to other KDE software, Kraft has disappointing little (known) users. Also contributors: Apart from very few brave developers who spent some time on Kraft, I am the only contributor. The reasons for that can be discussed in another thread.

What still keeps me motivated to work on Kraft is that the few users often tell how happy they are with it. And that Kraft really helps them to drive their business and save time. Also that they found with Linux and KDE a computer “environment” that really helps them reliably instead of facing them with scary stuff. That is really cool, and the best is that this recently happened more often than the years before.

That is what keeps me around with Kraft.

For the future, there are enough ideas: “Combined Kraft” which means using one instance of Kraft from distributed home offices, with ownCloud as sync hub, or an easy to use project management and of course a port to KDE Frameworks 5 to be able to ship for Mac and and and…

Well, it’s a pet project, and my day unfortunately does not leave very much energy behind for that currently, so don’t expect big movement, but be sure that there will be small steps.

Even on saturday morning, short after opening of the event, somebody from the orga team was showing up on the booth with chocolate for the volunteers, saying hello and asking if everything is in place for a successful weekend. A small detail, which shows how much effort is put into organization of the event.

As a result, visitors come to visit the event. It’s mostly a community centric event: Exhibitors are mostly representing FOSS projects such as openstreetmap.org, distributions like Fedora or openSUSE or companies from the free software market. [caption id=“attachment_428” align=“alignright” width=“595”] Morris in action on the booth[/caption]The majority of visitors are mostly interested in private use of the software. But, no rule without exception, we also had a remarkable number of people from companies, either executives or people working in the IT departments, who were interested in ownCloud.

Speaking about ownCloud, I want to say that it’s amazing to represent our project. People know it, people like it, people use it. In private, but also in professional space people work with ownCloud already or are planing to start with ownCloud. ownCloud already is the accepted solution for the problems that became so practical with the NSA scandal last year.

My talk with title A private Cloud with ownCloud on Saturday morning was very well received and went smooth. The room was too small, lots of people had to stand or sit on the stairs. It was a very positive atmosphere.

Something that changed compared to last year and the year before: Most discussions were around how ownCloud can be installed, integrated and used and not any more about which features are still missing or maybe also bugs.

So it were two very exhausting days, but big fun! Thanks to Daniel, Arthur and Morris for the work and fun we had on the booth, and thanks to the CLT team for CLT.

The ownCloud community released ownCloud 6 a couple of days ago. That was another big release and we want to celebrate!

Please, everybody who is interested in ownCloud, like to learn more, give feedback or just want to meet other people from the community, you are invited to show up at Coworking Space in Nuremberg, Josephsplatz 8, on december 18th, 6pm.

We will have a relaxed evening with a little discussion, maybe short demos, cakes and stuff, and fun. No heavy talks and serious faces!

We are looking forward to meeting you.

A nice proof that openSUSE has more than others was posted here by our friend Thomas, a convinced Debian user. He writes about setting up openSUSE in vagrant to be easily able to build (master build) the ownCloud Client for Win32 in it. Very easy and cool stuff. But that can be even easier without vagrant through this link ;-).

Btw, there is an appliance in SUSE Studio to ease experiments with vagrant with openSUSE as base. I haven’t tested yet, experiences?

I remembered awesome Bernhard was blogging about that topic recently. On that page one can find this link where raspberrypi images can be found. Oh, surprise, there is even a file from november 10th, so I downloaded that. People always recommend the latest stuff.

Following this Howto I quickly had the RaspberryPi running in my home network, surprisingly enough identifying itself as powered by openSUSE 13.1 :-)

Well, that was easy and far away from adventure which I was looking for. So I remembered that the cool kids on the block have an ownCloud server running on the RaspberryPi. Would that be as easy? There are no official packages for the Pi yet, so what could I do?

Well, ownCloud is noarch, because it is plain PHP. So I downloaded the two ownCloud server packages owncloud and owncloud-3rdparty from our ownCloud nightly build repository on OBS and installed them with

zypper in owncloud owncloud-3rdparty

I was (adventure!) ignoring all the warnings and stuff, what you should never do! Just for a test, before the coffee is cold.

After having started apache, what should I say? It simply worked. No need for antihistamine, all nice green around, and ownCloud running after having finished it’s setup page.

That really pushed me for the day! It was such a smart experience having that running within a couple of minutes, with absolutely no fiddling around. This is cool stuff! Thanks to Bernhard and all the other openSUSE guys for doing that!

My congrats for the 13.1 release! I really hope that people will understand (again) how awesome the openSUSE distribution and the project is, especially for the more nerdy folks! Really, you wanna run the Geeko these days.

Enough praise, now, maybe there is somebody who will help me in OBS to provide proper ownCloud packages for ARM? I am sure there is not much missing.

And if you want to run ownCloud on your “normal” PC, this is the repository of the latest stable version which we actively maintain…

The new release fixes a problem with the tarball of 0.51 which contained a wrong source revision. That did not cause any harm, but also did not bring the announced fixes. That was brought up by community friends, thanks for that.

Additionally another, actually the last known bug of Kraft’s catalog management was fixed. That was the problem that it did not work to drag sub chapters onto the top level of the catalog. That is working now.

Please update to the new version and help us with your feedback.

To do a step into that direction, here is a little script collection for you to play with if you like: the DAV torture collection. We started it quite some time ago but never really introduced it. It is still very rough.

What it does

The first idea is that we need a reproducable set of files to test the server with. We don’t want to send around huge tarballs with files, so Danimo invented two perl scripts called torture_gen_layout.pl and torture_create_files.pl. With torture_gen_layout.pl one can create a file that contains the layout of the test file tree, a so called layout( or .lay)-file. The .lay-file describes the test file tree completely, with names, structure and size.

torture_gen_layout.pl takes the .lay-file and really creates the file tree on a machine. The cool thing about is that we can commit on a .lay-file as our standard test tree and just pass a file around with a couple of kbytes size that describes the tree.

Now that there is a standard file tree to test with, I wrote a little script called dav_torture.pl. It copies the whole tree described by a .lay file and created on the local file system to an ownCloud WebDAV server using PUT requests. Along with that, it produces performance relevant output.

Try it

Download the tarball and unpack it, or clone it from github.

After having installed a couple of perl deps (probably only modules Data::Random::WordList, HTTP::DAV, HTTP::Request::Common are not in perl’s core) you should be able to run the scripts from within the directory.

First, you need to create a config file. For that, copy t1.cfg.in to t1.cfg (don’t ask about the name) and edit it. For this example, we only need user, passwd and url to access ownCloud. Be careful with the syntax, it gets sourced into a perl script.

Now, create the local reference tree with a .lay-file which I put into the tarball: ./torture_create_files.pl small.lay tree This command will build the file tree described by small.lay into the directory called tree.

Now, you can already treat your server: Call ./dav_torture.pl small.lay tree This will perform PUT commands to the WebDAV server and output some useful information. It also appends to two files results.dat and puts.tsv. results.dat just logs the results of subseqent call. The tsv file is the data file for the html file index.html in the same directory. That opened in a browser gives a curve over the average transmission rate of all subsequent runs of dav_torture.pl (You have run dav_torture.pl a couple of times to make that visible). The dav_torture.pl script can now be hooked into our Jenkins CI and performed after every server checkin. The resulting curve must never raise :-)

To create your own .lay-file, open torture_gen_layout.pl and play with the variables on top of the script. Simply call the script and redirect into a file to create a .lay-file.

All this is pretty experimental, but I thought it will help us to get to a more objective discussion about performance. I wanted to open this up in a pretty early stage because I am hoping that this might be interesting for somebody of you: Treat your own server, create interesting .lay files or improve the script set (testing plain PUTs is rather boring) or the result html presentation.

What do you think?

You find suitable download links as usual at: http://owncloud.org/sync-clients/

Version 1.4.1 is a bugfix release for the 1.4.0 version released a few of weeks ago which brought a lot of new features. This one solves a couple of problems that were coming up during the last few weeks. For example, the problem that the client lost its configuration (at least) on the Win32 platform when the machine was shut down is fixed. Also a lot of redundant uploads wont happen any more. And there are even more fixes, as the detailed Changelog rules out.

We thank you for your ongoing support and the good work on bugs that came up. As usual we are looking forward to your feedback. Please work with us in the Github bugtracker if you experience issues.

This is a bugfix release which brings a handful of useful fixes against bugs which were reported by Kraft users since the last release.

In the catalog view, now drag and drop is working to sort templates. Removing of sub chapters is also working now. A bug in the unit handling was fixed that picked wrong units in some cases. The path to document templates is not utf8 save.

As a new feature the address of the own company now can be picked from Kraft’s settings dialog also after first setup routine.

A source tarball can be downloaded from the Sourceforge Project, binary packages are on the way. Please also report bugs on SF.

Thanks for your interest and contribution to Kraft. If you want to support Kraft, please give feedback, spread the word or buy cool stuff.

Danimos Visual Guide has outlined the new stuff in the release already, so no need to repeat it here. You should install and try it, that seems to be the opinion of many people who tried it.

Also people who shared their critical view on the client very publically in the past are much more pleased now with 1.4.0. One example is a recent blog post on BITBlokes. It is a blog about all kind of topics around FOSS. I regularly read it and often share its opinions. He concludes very positively about the 1.4.0 client.

It is good to see the positive feedback overall. That shows a couple of things from my engineering point of view: The concentrated work we continously do on all parts of ownCloud pays off. That is obvious of course, but still nice to see. And our (also obvious) actions to improve code quality such as the consequent use of continous integration, code reviews and such helps to improve quality.

“People are always excited if releases come with GUI changes!” I heard people saying. Well, maybe, but that’s not the whole truth. It also proves for me again is how important UI design and UX is. Me as a knee-deep-developer have an interesting relationship to all UX topics: I always have an opinion. Often a strong opinion. But the results coming out of that have not always been the, well, the most optimal. Very fortunate on the client we work together with our UX guy Jan and the positive feedback also shows how good that is for the software.

But enough of release pride. There is more work to do: The bug tracker is still not empty, the list of feature ideas is long. We will continue to focus on correctness, stability and robustness of syncing, performance and useful features and work on a version 1.5 for you.

These are a couple of concrete points we’re focussing on for 1.5:

1. we already merged the client code on the new upstream sync version in git.
2. performace improvements through further reduction of the number of requests and more efficiency in database operations on the client.
3. we are working on a new propagator component that allows us to do the changes mentioned in 2 more easily.
4. File manager integration, which means havingn icons in Explorer, Dolphin and friends.

A more detailed list can be found at github.

Thank you for all your help and support. It’s big fun!

First and foremost, beta 2 fixes a severe bug on the windows platform which was the root cause for a lot of other bug you might have experienced, such as infinite sync loops and similar. Please urgently prefer the beta2 over beta1 for testing.

Other fixes went into polishing of the new progress display, the sync protocol window which is not sync-folder centric any more and many other places.

We encourage thorough testing of this version, especially the new features. In addition, we would like to ask you to check the open issues in https://github.com/owncloud/mirall/issues and tag them with the “1.4” tag if they are still reproducible.

Thanks a lot for your enthusiasm. That helps us to improve ownCloud.

The new release contains a lot of features and changes me and my collegues worked on during the last 18 month: First we added the ownCloud module to csync upstream, so that csync now is able to sync local directories to an ownCloud server. The ownCloud client works on platforms Linux and MacOSX and Windows. That required a lot of (tricky) changes to the csync platform which we carefully backported to csync upstream.

There are much more changes, such as: More compilers supported, a new logging framework, a new base lib to do unit testing, removal of not so common dependencies and other infrastructure changes. But also on the feature side there are more improvements, mainly to focus on a more easy and broader use of the csync library that allows to embed the sync functionality to various backends into other applications.

I am very happy that we kept our promise to contribute a lot of our changes back to upstream. Primarily that is the way to go in the open source world, you will say. But it was distressing to see how quickly people were whispering stuff like “Well, wouldn’t it be easier if you do a fork? Remember, you have to come to good results quickly for the company, why bother with an upstream project?”

For me, that wouldn’t be like companies in FOSS world should behave, and I know that on the long run, it would not be beneficial for the company either. So very good that the relevant people supported the idea of going with upstream from the beginning.

Does that mean that csync 0.50.0 and ocsync as shipped with ownCloud are the same now? No, unfortunately not. We did more changes on the ownCloud branch which need to be reviewed and probably won’t fit to csync the way they are now. More work to be done, but we’re on the way!

Many users of the ownCloud Client were asking for sync progress information, in fact there was none at all until today which is a bit boring. The reason why we hadn’t it was simply that csync, which is the file synchronizer engine we use, did not have an API to hand over progress information of an actual up- or download to higher levels of the application.

We implemented two callbacks in csync: One that informs about start, end and progress of an up- or download of an individual file. Another one processes the overall progress of the currently running sync run, indicating for example that eight files have to be processed, current is file number four, and x of the overall sum y bytes have been processed already.

That information is passed into a singleton class called ProgressDispatcher in the client code, where other classes can connect to a signal providing that information. That comes handy as we need the information at different places in the client GUI. [caption id=“attachment_299” align=“alignright” width=“399”] Sync Progress Display[/caption] The screenshot shows the first and probably most detailed implementation of the progress display in the sync accounts details which are part of a new settings- and status dialog.

The visual appearance was worked out by our interaction designer Jan. We always have to argue a bit because obviously I am the greatest interaction designer around ;-) but well, finally we do like Jan says and the result is great IMO. It’s great that we have him for ownCloud as that guarantees that our project is not drifting too much onto the geeky techy side which (I heard) is scaring off some users…

Hope you like it! You can get a preview in the current nightly builds (win, mac, Linux) of the client! Please report bugs as you find them, thanks.

We revisited this problem and will come up with a better solution.

The key changes will probably be

• Conflict files will never again be generated on the server. Even if we are in a conflict situation, we will download the file and keep the conflicting version only on the client. This enables us to detect false conflicts.
• The current way we handle a system time difference has to be changed. We wont adjust the file mtimes of files in the file system any more with the time difference between the client and server. That way we do not suffer from floating time differences any more. For the decision of which version is more recent, we will still consider the time difference.
• We will use a very quick request like OPTION to get the servers time setting to the client. That will allow to calculate the time difference between server and client more accurately. It’s needed to decide which file is more recent.

Read this as a note to self, yet we feel very well fitting into Berlin round may 1st ;-)

Photo credit: 96dpi / CC BY-NC

Here is a small overview of what went into ocsync master (actually the dav branch) and into mirall master.

Note that this is alpha state software, but it’s already available for testing from our nightly builds at the ownCloud nightly repository.

If you enable the nightly package repository on your system, please be sure to use the machine for testing purposes only. Windows and Mac nightly builds can be found here to download.

csync: Big File Chunking

Handling big files over HTTP is difficult, if files grew bigger than a gigabyte, we faced multiple problems. To deal with that, we thought of splitting up big files and transmit the parts to let the server finally assemble the file again once all parts are there.

We wrote a little library to implement that, called httpbf. It is now integrated in the current development branch. You will see files big files split up into parts for transmission to the server. With that there should not be a filesize limit to work with any more, except the limits we have from the operating system etc.

Up- and Download Resume

With big file chunking, we also introduce up- and download resume. If a file transmission was interrupted and is started over again later, only the missing parts are going to be pushed again. However, we’re not yet transmitting only changed chunks for existing files yet, just to answer that question here before it comes up. That will be subject of more enhancements.

Source File Verification

Another problem with big files is that if one gets copied into the synced dir on the local machine, the copy is not finished before the sync acutally starts. That can result in useless transmissions because inconsistent data is synced.

We added code to the httpbf library that checks for the integrity of the source file. When the first chunk is transmitted, it is checked if the file has still the size and mod time from the beginning. If not, the library waits for two seconds and starts over with a check again. That way, it waits until the source file has stabilized. A final check if the source file hasn’t changed during upload is done for every file when the upload is finished.

CSync Context Reuse

There is the very unpleasant problem on some distros (namely Ubuntu and Fedora) that the ownCloud Client wastes file descriptors. We figured that this is because the system libraries on that platforms fail to unload the csync ownCloud module properly.

Version 1.2.5 contains a quick fix for that, but the real fix is to reuse the csync context within mirall and not build it up and down for every sync run as we did until now. So we developed the function csync_commit which commits the results of the sync run to the database and leaves the csync context intact for the next run. That way we just load and unload the ownCloud module on time and not every 30 seconds. The csync patch already went upstream.

Mirall: New Setup Dialog

There is also the new setup dialog in the nightly builds: It is much more simplified compared to the old one, user only needs to add the url and user credentials and thats it. Also, it syncs the whole ownCloud to a local folder, just like other boxes do. That is one of the most often heard enhancement requests, and here we go.

That all I think would qualify for a good 1.3.0 release, but we need more time to fine tune,polish, test and bugfix. But we would appreciate if people could help test. Please report bugs in the client’s bugtracker. We need to find bugs prior the release, and that happens with your help!

Have fun & Thanks a lot for testing!

We just release ownCloud Client 1.2.5, hurrah! It’s a small bug fix release, but it will make more people happy users! It fixes three ugly crashes we found with the help of encouraged bug hunters. One of them was an race between our thread which runs csync and the Qt SSL stack we’re obviously using. That caused a crash which happened after quite some run time, or also never… Hard to reproduce and debug, but finally Danimo and his mighty friends were able to nail that.

The new client can be found through it’s download page, here is the Changelog.

If you find issues, please help us to improve ownCloud by commenting or reporting the bugs on our bugtracker. Thanks!

The nightly builds of the client for Windows and Mac and the source balls can be downloaded from the nightly directory on our download server. They are named after the last release, followed by a date time stamp, which is simply a concatentation of the year, month and day the nightly was built on. For linux we maintain a so called nightly repository in the openSUSE Buildservice which builds for various distributions. That can be added to the linux package management system and that way every morning, an update is there for you. The ownCloud server is built only in the nightly repository from this daily source.

What are nightlies for?

To test ownCloud is a very difficult task. ownCloud is a complex system consisting of variuos parts (anybody thought ownCloud is just a “LAMP web app”?) which supports a huge variety of environments. That multiplies to a huge variety that needs to be tested. We had issues with that in the past and as a result, providing easy access to ready-to-run testing versions is one important action we are taking to improve quality as that enables more people to participate on tests.

Of course nightly builds are not released software, nor got they extensive testing. Nightly builds must never be used on production installations, please, really don’t do.

But maybe you have a test server or virtual machine somewhere in spare. So it would be great if you could install a nightly version on that, combine it with a nightly built client, and see if everything runs as you would expect it.

Another great purpose of the nightly builds is to use it for bug verification. In client development we often comment on bugs with sentences like this is fixed now, please verify with the current nightly build. That way bug reporter get a very quick way to verify the fix of their issue and developers can be sure that their fix really is sufficient.

How are nightlies built?

We are doing the nightly builds through an instance of Jenkins, a continous integration server. With a little script we wrote Jenkins feeds the OBS every night, so this is completely automatted. But more on that in a subsequent blog.

Let’s meet Wednesday, February 27th, 6:00 PM at the Nuremberg Coworking Space.

Talks (short, don’t be afraid!):

• ownCloud overview by Holger Dyroff
• The updated ownCloud 5 User Interface by Jan-Christoph Borchardt
• Next steps for the ownCloud Desktop Clients by Daniel Molkentin

The evening closes with a get-together, your chance to meet the team and discuss in a relaxed atmosphere.

Everybody is welcome!

What is interesting about the release from a more technical point of view? Here are a couple of examples.

One of the things which often was complained about was the performance of the client. Performance is a very broad term, so we have to examine the details: Many people felt like its a performance problem that former clients polled the local file system for changes on the MacOSX and Windows platform. That was recommended from other projects which have experience with going through large file collections. QFileSystemWatcher seemed not to be designed for this usecase.

Polling was ok for desktop computers with up-to-date hardware, but on devices running from batteries this was a energy drain. Good that we could fix that with 1.2.0 using native implementation of change detectors on Windows and MacOSX. They detect changes within a file system tree. If a change is detected, a sync run is started. But note, this is only for local file systems. Detection of changes on the server is a different story which has to be told as well.

Another performance problem was within the file upload which is done by a HTTP PUT request. Month ago (pretty after my start at ownCloud) I implemented it using a tmp file in between. That means, the source file was copied to a tmp file, and from that it was processed to the request body. Improvement was needed and we changed the code to directly read from the file descriptor of the opened source file.

Another thing that was improved with 1.2.0 is the error reporting to the user. Former versions of the client sometimes provided error messages which were not really accurately describing the problem. The reason for that was that csync uses errnos (yes, the ones from errno.h) to name errors as csync maps everything to a POSIX file interface. That surely works as long as you’re on a kind of file system. But it’s hard to map HTTP communication problems onto that. So we decided to add our own “custom” errnos and enhance the whole idea to use these to describe problems. That works more accurate now.

The next things on the list are for example a more convenient setup dialog for the client. Also we will get away from the kind of hard coded target file name on the cloud. A better network recognition will also be next as well as better handling of big files. And more…

Thanks a lot to all who helped to get 1.2.0 finished! It is big fun to work in such a great community :-)

The release of version 1.2.0 is planned for the next week if things go smooth.

[caption id=“attachment_219” align=“alignright” width=“595”] New Sync Protocol Dialog[/caption]

In particular, the the following improvements were added:

• Proxy authentication fixed (Basic auth, NTLM will not yet work)
• The status dialog now provides statistics on the last sync run (via the info button). It will tell in detail which files have been synced, added or deleted.
• Client will go offline while the server in in maintenance mode (feature available with ownCoud master only)
• Improved SSL Certificate acceptance
• All sizes of the new icons are available.
• Support files > 2 GB on all platforms for uploading.
• Fixed some minor memory leaks and again saved some server requests through optimizations.
• Improved error reporting to the user.
• Remove legacy theming support.

We would appreciate if you give this release a test ride. Note that because it is beta you should make extra sure to have have backups of your data.

If you want to give feedback, please use our mailing list for general discussion and the issue tracker for bug reports. Please read our new guidelines on bug reporting before!

Download Links:

• Windows, 32/64 Bit: http://download.owncloud.com/download/testing/owncloud-1.2.0beta2-setup.exe
• Mac OS X ( >= 10.6, 64 bit): http://download.owncloud.com/download/testing/ownCloud-1.2.0beta2.dmg
• Linux: http://software.opensuse.org/download/package?project=isv:ownCloud:testing&package=owncloud-client

Sources:

• Mirall: http://download.owncloud.com/download/testing/mirall-1.2.0beta2.tar.bz2
• OCSync: http://download.owncloud.com/download/testing/ocsync-0.70.1.tar.bz2

Have fun!

This is the first build with the new things we did in Berlin a couple of weeks ago, you will

• discover that there is much better error reporting if something goes wrong.
• probably feel like it syncs faster, yes faster.
• see that there are less HTTP requests to the server for a single sync run.
• don’t see any issues with MacOSX and funny characters in filenames any more.
• recognize a new icon set, which is not finalized yet (actually not all sizes are there, thats why the status dialog looks a bit funny) but we thought its nice to already add it to the beta. It should fit nicely into your operating system environment.
• realize that this client comes with a cross platform file system watcher on clientside, so no polling any more.
• have your password stored in a secure keychain on all platforms since we added qtkeychain to the client.

Maybe there is more, but we thought that’s already a nice beta release.

Please find packages for MacOSX, Windows and Linuxes. Note, not all packages are finished yet. If the one for your distro is missing, please come back later, or even better - speak up at packaging@owncloud.org and help fixing :)

Of course you also should note that this is an early beta and you would not want to use it without a good backup of your data and only on your test account without important data.

We would appreciate if you let us know your experience on the mailinglist. If you find problems, please report it to the client’s bugtracker mentioning client- and server versions and at best with useful logs.

With that we are happily vanishing to spend some time away from the computer, looking back on a very exciting and very busy year, working on an interesting topic with a lot of nice people.

Thanks and best Season’s Greetings!

The most important change is that now multiple tax rates are supported within one document. That means that within e.g. one invoice items without tax, with reduced and full tax rate are supported. The sum calculation takes that in consideration properly. That enables Kraft for even more usecases.

Apart from that, a lot of bug fixes and improvements happened, read more details on the Kraft Homepage.

Sources and binary packages for various Linux distributions are available from the Download Page or will appear there soon.

This release is nice progress for Kraft and I hope you have fun and success with the new release. Thanks all who contributed to Kraft! I am looking forward to getting your feedback!

We thought that it might be fun to work together on csync, our sync engine under the hood of the ownCloud client. There were some issues that should be fixed and on the way we cleaned and improved quite some code in csync.

Here are some things we worked on:

• We added a function that lets the program that uses the csync library pass arbitrary module parameters to the backend module. That way its more easy to steer the behaviour of the ownCloud modules from the calling app.

• Error handling was improved, ie. if an http error happens, csync works errno based error reporting. We added custom errnos because not all error cases with http can be mapped to system errnos.

• Formerly the csync ownCloud module was spooling files through an additional temporar file on client side. That step is skipped now which results in performance improvements as well as in more clean code.

• We were able to reduce the number of HTTP requests that go over the wire even more. For example to check if there are changes on server side, now there is exactly one http propfind required. Also if files have to be synced, we could save some HTTP requests by improving caching of some requests.

• Andreas recently changed the logging system in csync upstream master branch. We merged that back and now do not longer need the log4c framework. One build dependency less and a nice new logging framework.

• Other bugs were fixed, such as a potential crash if a folder as deleted during it is synced, SSL handling shortcomings, code streamlining in handling compressed data streams and more.

• We finalized a patch that uniforms the utf8 representations of characters over all platforms. That will fix problems we saw especially with MacOS and special filenames.

Ah, yes, we also did other things, more related to the ownCloud client. Danimo managed to implement a cross platform filesystem watcher class that is able to fulfill our requirements. That obsoletes polling for changes on the local file system, one of the most popular enhancement requests.

And finally there now is a API in csync thats reports file transmission progress if a callback is installed accordingly. So the client hopefully soon will tell ya what it’s doing for you. Also appreciated I guess…

Last but not least we added code to use QtKeyChain, a cross platform password storage library that stores password encrypted. For example on Linux QtKeyChain connects to kwallet. QtKeyChain was provided by Frank Osterfeld, thanks a lot for that contribution.

Quite some stuff for a short week, note that stuff that fills a short line in this blog can be quite nifty to investigate, implement and test. Not everything is stable, polished and properly integrated but it was a great and productive week. The next release of ownCloud Client will be a nice one.

And sice you can not always work, we had a nice dinner at a very cool italian restaurant. We met with other ownCloud employees located in Berlin, Arthur and Georg. Fun :) And Berlin, yes, a great place to be, but finally I appreciated to arrive back to my snow covered home.

Many thanks to Olivier and Markus for hosting us and for the nice week.

I gave a talk about ownCloud on Saturday where I tried to show some technical details. That could have gone even deeper was the feedback some people gave me so I promise to show more of for example app development next time. Apart from that (and apart from the fact that laptops, projectors and Linux still do not play well) the talk went ok and was well received.

The conference is a joint conference of actually four conferences: The openSUSE Conference, the Czech Linux Days, the Gentoo Summit and the SUSE Labs Conference. Over the weekend it was hosted at the Czech technical university of prague and that was a great venue. Very inspiring that over the whole foyer of the building intersting art sculptures were shown. Unusual for a technical faculty but a great thing. The building is modern and large enough for all the talks and workshops but as a result of that it was sometimes a bit difficult to keep the overview where happens what. Especially because the start- and end times of the talks were not kept in sync over the tracks.

I found it is a different atmosphere on the event compared to the openSUSE conferences of the last years which might have been more focussed around the openSUSE project. This one felt more like an general FOSS event. That is not bad, just different, and since change is obvious and good I look forward to seeing how this influences the openSUSE project in the near future.

I like to thank all involved on the organizing teams for their successful hard work, have a lot of fun for the next two days, I had a lot during the last two days :)

This blog will shed some light on the details. I apologize, it’s a long read.

Time Issues

ownCloud Client versions 1.0.x worked with csyncs traditional way of using the file modification times to detect updates between the two repositories that should be synced to each other. That works fine and conforms to our idea to ideally not use any other metadata in syncing than what the file system has anyway.

However, there is one drawback which we all know from daily life: If at least two parties sync on time its important that all clocks are set exactly the same way. Remember good crime movies where a bank robbery always starts with a clock adjustment of all gangsters? We have exactly the same in ownClouds syncing: All involved have to have the same time setting, otherwise modification times of files can not be compared reliably.

There are solutions for computers to set the exact time (like ntp) so in general that works. However, in real life scenarios these are not reliable because either people do not have them started on the system or because the daemon updates the time once in a while and in that time span the clock skews already too much.

Users all the time reported problems with that and other experts continued to advise that we never get around that problems if we don’t change something fundamental and go away from pure time based syncing.

Well, we did that with our csync version 0.60.0 which is the sync engine for ownCloud Client 1.1.0.

An Unique Id

Now, every file and directory inside a sync directory has an unique Id associated. The idea is that the Id changes if the file changes. So in the sync process the need for a file update in either direction can be computed by comparing the two Ids of the file. If the id has changed on one repository the file was changed there and needs to be synced to the other side.

The Ids are generated on the ownCloud server and one challenge for the client is to always download the correct Id of a file. The Ids are just random tags for a file version. It is not associated to the file content as MD5 sums would be. Actually it was a frequent advise to use MD5 sums or a similar approach which digests the files content to detect updates. That would have come very handy because that means comparing file contents directly and, more important, it’s reproducable on either side. Also the client would have been able to recalculate the MD5-Sum of the local files and would not have depended on a local database with Ids that were pulled from the server before.

But we decided against hashes. Calculating MD5-Sums is costly in terms of CPU and time, especially for large files. The CPU problem is small on clients, but not on servers where a lot of clients connect to. Even though the sums can be calculated during upload, the problems remain for the case where the server does not see the upload stream, think of the “mount my Dropbox” case.

For files on the ownCloud server, the Id is always updated when the file gets updated. On the client side the last Id of a file is in the client database. It is invalidated in case the files modification time changed meanwhile to detect local changes.

Change Propagation

Another remarkable change in the 1.1.0 client is that change events in the file tree propagate up to the top directory on the owncloud server, ie. if a file changes in a directory, the id of the directory changes as well as the one of its parent directory etc.

That means that to detect if a file tree has changed, it’s enough to check the top most directories Id. If that has changed, ok, than the client needs to dig deeper, but in the not so rare case that nothing has changed, the one call is enough to detect that. That dramatically lowers the server load with clients because instead of digging through the whole directory structure what we did with the 1.0.x series it is a few requests now.

CSync and ownCloud for Success

These are very intrusive changes to csync. For example, we had to add two additional fields to the database, add code that is able to build a representation of the local file tree from the database and make csync query for the file Ids from the server if needed. Deep under the hood the updater, reconciler and propagator code needed changes to work with the Ids. All these changes did not go back to csync upstream yet.

To not conflict with the upstream version of csync we decided to rename our csync version to ocsync. But: This is a temporar solution for the time we need to catch up with upstream again. That will take a while until everything is sorted again but we will work on that.

I am are very excited about the new version of csync. But obviously there are other changes in the ownCloud Client 1.1.0 which will be subject of another blog post.

The OBS is a great system as it does not only build for various Linuxes but also lets the users download the binary packages utilizing a huge mirroring infrastructure. All for you, all for the good of free software.

And there is even more: OBS offers a nice download page which can either be embedded into a projects webpage or be linked from it. Depending on the settings of the OBS project, for example the number of Linuxes it builds for, the page is adopted automatically. So you never again have to deal with lots of download links to the projects repository downloads. See the ownCloud Client Package download page as an living example.

Now the page had a shortcoming for apt based distributions. I became aware of it as the problem as well as the solution to it was nicely pointed out by this blog: Apt wants to have the key before it trusts the download source. OBS generates the key but its not obvious how to import.

This morning I told one of the brave OBS developers to the blog post and already now the solution is online. From now on, the download page also shows the exact command line to import the key. Isn’t that great how quickly things become better if a couple of people do a little bit work together?

Thanks a lot, guys, really cool…

Subject of the BOF will be the integration between KDE and ownCloud. As onwCloud originated from the KDE project we should meet and think together again how we can provide useful integration for the users combining the power of both technologies.

After a short overview of where the ownCloud project is going and what the recent achievements are (if participants are interested) I think we should concentrate on the technical level.

There are a couple of rough ideas already how we can integrate on a technical level, such as

• Integration of the ownCloud data storage in the KDE user experience.
• KDE features for the ownCloud Desktop Client like KWallet integration or KDE SSL certificate management.
• Synchronisation of PIM data like contacts, addresses, bookmarks or RSS feeds
• a syncing API for application data to be synced in KDE applications.
• Syncing of KDE application configurations.
• Plasma integration of ownCloud/syncing client.

I am sure there will be more we can come up with. Let me know upfront if you want me to put it on the list or even better show up at the BOF in person.

Along with that I am happy to let you know that we also released the ownCloud Desktop Client in version 1.0.2.

It is a maintenance release with a couple of important bugfixes, such as the cross platform filename encoding problem I was already talking about. Apart from that big blocker a couple of smaller, but annoying problems were fixed. Also the GUI was polished, text changes here and there and a new icon set that looks more cool and more like ownCloud.

Version 1.0.2 is also the version that is released on all big desktop platforms the first time. Now we also offer a dmg for MacOSX as well as a Windows Installer and packages for the major Linux distributions. Have fun!

If you want to talk about ownCloud, file synchronization or such, we have a booth on Linuxtag in Berlin and I will be there. I am looking forward to meeting you.

This bug took me on an interesting (well…) journey into what cross platform really means.It is about a complex thing like filenames containing special characters, my favorite example is the directory “Übergröße”. Directories and files like this could not successfully be synced with the latest client.

Windows does not utf8 by default. As far as I understood it, it does a local encoding by default. If you use normal C functions to access the filesystem, something like readdir or so, you get the names in that encoding. But, because that caused problems in some point, there is another set of functions which often start with _w (for example _wreaddir) and that has an interface to deal with wide characters. This is the way to internationalization.

So what needs to be done is to use the wide character implementations everywhere where you deal with filenames (in my case). That, in turn, requires to use wchar_t instead of normal char. But that again requires a lot of #ifdef _WIN32, if its done the naive way at every place it needs to be done. I learned that typedef helps here and a meta type is defined called _TCHAR. And also for all the needed functions, meta functions are defined which are replaced by tricky defines depending on the platform. For example _treaddir becomes readdir on linux and _wreaddir on win32, while _TCHAR becomes wchar_t on Win and char on linux. The code is written using the meta types to not poison it too much with platform ifdefs.

But that is only the first step. Wide character is not utf8! And since utf8 should be used within the software all over, a conversion from wide character to utf8 and back is needed whenever the file system is accessed. Thankfully from csyncs nature, these places are quite nicely concentrated.

All that combines into a larger patch. And now the “Übergröße” folder can be synced correctly from Windows to Linux and back. Great.

What can we learn from that? Well, easy, and once again: If you have no Qt, you’re alone. Better don’t let that happen. If you have Qt, be happy and aware of what it does for you :-)

Only a couple of weeks after the first release we could come up with a quite long changelog, larger than for a maintainance release.

First and foremost, oCC now supports SSL connections and does not store passwords any more, well, in clear text in a config file. Still there is no strong encryption for the password in (as that’s a non trivial if not impossible thing) but its not clear text any more, so accidents should not longer happen. For those who really care and put security over convenience there now is the option to not store the password at all but provide it at startup. The whole password storage will get more attention soon, I did some work on that already, also using kwallet, more on that in another post.

The whole start process of oCC, which is more complex as it might seem as libcsync expects a nice environment with a verified connection to the ownCloud, was cleaned and matured. I had to learn about QNAM but now we seem to be friends.

Efforts also went into the following areas:

• Error handling: People got confused about error messages from oCC which were not accurate here and than. I added more fine granular error reporting to libcsync so that oCC now can give even more helpful error messages.
• MacOSX: On the sprint weekend in Stuttgart, we made good progress on that platform, will also soon get a release. Domme got the most out of cmake to get dmg image building going, thanks, you rock!
• Qt 4.6: As usual one has to learn that current distributions ship not so current Qt versions. Thanks to Thomas, he put quite some effort in porting back to Qt 4.6 - so that Debians friends can also use oCC now.
• Packages: Check out the community repo on OBS: We now can provide packages for way more platforms including Ubuntu and Debian. Still your help would be appreciated in maintaining.

Speaking about the recent sprint in Stuttgart: It was great to be there, ownCloud has an open, friendly and welcoming community in which you really feel the spirit of working on something new and interesting with cool potential. I really enjoy being here,. Thanks guys for all your work and feedback :-)

Linux and Windows development should go in parallel as easy as possible.

Edit: Please also refer to the official build documentation.

Building CSync

To build mirall, csync must be built first. CSync is hosted in its upstream git repo and there is also my development branch which holds the latest changes.

Clone the csync branch into a directory. In parallel to the cloned csync dir, create a new directory buildcsync as a cmake build dir. Change into buildcsync and call cmake like this:

cmake -DCMAKE_BUILD_TYPE="Debug" ../csync

and watch its output. You probably have to fulfill some dependencies, make sure to install all the needed devel packages. You will need log4c, iniparser, sqlite3 and for the modules libssh, libsmbclient and neon for the ownCloud module. Once cmake succeeds, call make to build it. So far relaxed for Linux.

To build csync for Windows, there are a couple of possibilities. The one I chose was to cross compile with mingw under openSUSE. That way I can build for all on one devel machine under my prefered system.

For that, I installed the cross compile and mingw32 packages from the openSUSE Build Service, which really demonstrates power here. I used the mingw repository. Kudos at this point to Dominik Schmidt, a Tomahawk developer, who helped me a lot to set all up and to all people who work in OBS to maintain the mingw repo.

Basically the cross compiler and libs (eg. packages mingw32-cross-gcc, mingw32-gcc-c++ and mingw32-cross-cpp) and the dependencies for the software to build have to be installed from the mingw repo. An action item is left to dig which in detail.

After installation you should have some mingw32-tools such as mingw32-cmake which should be used to build for win.

Now create a directory win and within that again buildcsync. In there, start cmake with

mingw32-cmake -DCMAKE_BUILD_TYPE="Debug" -DWITH_LOG4C=OFF ../../csync

That should do it. I did not find log4c for Win32, so I disabled it in the cmake call. Now build it with mingw32-make and see if it creates a dll in the src subdir and csync.exe in the client dir.

Building mirall

For mirall, it works similar. Mirall uses Qt and is C++, so again a lot of packages to install. Make again sure to have the mingw32-qt packages, for example mingw32-libqt4-devel and more.

However, there are two caveats with mirall:

• the current development state of mirall needs the latest devel version of csync which we just built. I tweaked the CMakefile that way that if the mirall- and csync and build* folders are in the same directory, the csync is found by mirall cmake in the parallel dir. So I do not have to install the devel version of csync in my system.

• to build mirall for windows, it must be made sure that cmake finds the mingw32 Qt tools like moc. Since there is also the Linux moc in the system, this can confuse. Domme pointed me to a script that sets some variables correct values to prevent mixing: cat ../docmake.sh # %_mingw32_qt4_platform win32-g++-cross export QT_BINDIR=/usr/bin export BIN_PRE=i686-w64-mingw32 /usr/bin/mingw32-cmake \ -DCMAKE_BUILD_TYPE="Debug" \ -DQMAKESPEC=win32-g++-cross \ -DQT_MKSPECS_DIR:PATH=/usr/i686-w64-mingw32/sys-root/mingw/share/qt4/mkspecs \ -DQT_QT_INCLUDE_DIR=/usr/i686-w64-mingw32/sys-root/mingw/include \ -DQT_PLUGINS_DIR=/usr/i686-w64-mingw32/sys-root/mingw/lib/qt4/plugins \ -DQT_QMAKE_EXECUTABLE=${QT_BINDIR}/${BIN_PRE}-qmake \ -DQT_MOC_EXECUTABLE=${QT_BINDIR}/${BIN_PRE}-moc \ -DQT_RCC_EXECUTABLE=${QT_BINDIR}/${BIN_PRE}-rcc \ -DQT_UIC_EXECUTABLE=${QT_BINDIR}/${BIN_PRE}-uic \ -DQT_DBUSXML2CPP_EXECUTABLE=${QT_BINDIR}/qdbusxml2cpp \ -DQT_DBUSCPP2XML_EXECUTABLE=${QT_BINDIR}/qdbuscpp2xml ../../mirall

With that setup I can build both the Linux and Windows version quite easily. There is still a lot to be solved, such as automatted packaging and such. CMake as usual is a great help.

Last week I have been on CeBIT for a day, which is a huge and busy event. We had a nice preview running there with syncing between the web client and the desktop clients on Windows and Linux, as well as on an Android device. That already worked quite ok, nevertheless I got quite some bug reports since then. And I still have to work with csync upstream to get patches in there before we’re ready for a public beta.

This weekend I will be at the Chemnitzer Linuxtage together with friends from the openSUSE project. I will show some ownCloud syncing there as well, so if you like show up at the openSUSE booth for a demo and a chat :-)

That involved basically three things: A csync module, a patch in ownCloud and work on mirall.

For csync, there needs to be a module that handles the connection to ownCloud, for code see [1]. ownCloud has a WebDAV interface, and its used for this. Csync requires the the implementation of a couple of POSIX functions such as opendir, readdir, closedir, stat and these kind of things. I did that using WebDAV equivalents.

Unfortunately one method is needed for the sync algorithms which has no equivalent in WebDAV, which is utime. utime sets the modification time of a file. csync needs that to fix the time after a file was copied to the other side.  For that, I made a little patch to ownCloud to allow a PROPPATCH method on the modified time of a resource. That makes csync happy and working, but that also means that the module is not a general csync webdav module but a csync owncloud module. Named it accordingly.

The first action I started on mirall this time is a bit cleanup. First, I introduced a class  to move all the code which loads, edits and monitors the folders out of the main application class, which was long overdue. That separates the sync folder management in mirall from the GUI stuff. Furthermore I added a few helper classes such as a config file class for mirall. Ah yes, a theming class was also added to handle the future two faces character of mirall easily, as said before mirall should either appear as traditional mirall client, or as ownCloud client. Duncan and me agreed to build that from one source, the theming class supports that.

Last but not least I polished the wizard in which a user connects to his ownCloud and added a test call that immediately evaluates the url and credentials. For the curious, the code is in my work repository but I promise to move it to the ownCloud repo on gitorious soon.

So far for now, if you have suggestions for me I’d love to hear them.  Thanks :-)

[1] Klaas’ WIP branch csync