Chronicles of DIY coding and typography

While I was at this, I’ve changed a bit the CSS of the development version of the site. I’m pleased to offer some screenshot for comparison. It looks more 201x now

Old and new front page

Text page

Looking at the “Book” feature of wikipedia, I was wondering if the library too can have something similar… or even better!

http://en.wikipedia.org/wiki/Special:Book

I’ve started working on some code, and so far it’s doing well… well, a bit messy, but definitively doable. Here there’s a preview of the rough interface I’ve built so far:

Basically, the user can choose its favorite texts, merge them, choose font and paper, and have the PDF ready in about 1 hour. I’ve thought about preparing them on the fly, but due to the not infinite server resources, it would expose too much the library to denials of service (imagine only 10 users wanting a custom PDF at the same time, the server would quickly become unresponsive).

Much is still to do: documentation for the user, the backend, the ConTeXt styles, and so on… So stay tuned!

Well, it’s time for another torrent release, with the fresh PDFs produced by ConTeXt. The DVD iso (2.1 Gb) is ready, has been tested, and now has been burned on 30 physical DVD, to be distributed at
the Zagreb bookfair

The printed stuff is also ready.

Once I’ll come back home, the torrent will be release, shipping the same ISO of the DVD. Hurray!

As today, December 22, 2010, http://theanarchistlibrary.org runs the new engine, using ConTeXt as typesetting system. Hope you’ll enjoy.

P.S. To celebrate I’ve printed out Nightmares of Reason (in its A4 imposed variant).

Hello there.

I’ve not updated this blog for a while, but I was very busy coding and polishing and testing. Now I’m very, very close. While at the first glance the site will look the same, and the PDFs will look very similar, under the hood a lot of things changed.

Some other new features:

1) multilanguage support. This means that is possible to build a sister site in no time (just translate the interface and some tuning via a configuration file written in basic perl, something like $setting="value"). It’s also possible to use the library format (as described in the guidelines) as a standalone converter. And the library format is just a subset of html, just using blank lines as paragraph delimiter instead of the tag <p>

2) microtypography. The new PDFs, even if to the untrained eye looks more or less the same, have their quality really boosted. Btw, a side effect is that the copy and paste from the pdf will work better (no more ligatures shown as weird characters).

3) It’s possible to define new pdf formats with no pain. I was thinking about a web version (horizontal) and a “POD candidate” (Print on Demand candidate, for people that wants an hard copy printed by professional people) which should replace the unimposed a4/letter. Or even to be added. I don’t know, but at this point the possibilities are infinite.

4) Web interface: no changes for the end-user, minor changes for us; there’s a news editor (locked for the rest of the world) and a spool manager, to avoid the use of the shell just to move a couple of files in an archive directory.

5) Support for file renaming. Renaming (due to titles that get fixed) is/was a pain, because it leaves behind broken links and outdated files. I’ve added a “router” that searches in a file (which is possible to compile in no time thanks to the revision control) in the root of the “alibrary” git, and redirect to the correct links. As fallback, it will route to the search.

6) Some other tags support, including <em> and <br /> in the title and in the author fields (keeping the output formats standard compliant). Basic tables are now supported. Images are handled better. The purpose is to get rid of most of the <pre> </pre> shit.

7) General code cleaning that nobody cares about.

8.) the tag cloud will pull from both the authors and the topics, picking the one with more texts. Tags with names should be at this point deprecated. This will help keeping the tags to a sensible number.

9) There will be a new field, with “Author for sorting”. It should be set to Last, First, to have a proper index.

After a week of reflection, I’ve decided to merge the two repositories of the library code. I’ve pushed it to gitorious and I’ve updated the link.

The development will happen in the “ng” branch.

It turned out that I’m sick of testing against over 1200 files. So with a little of patience I’ve compiled a dozen of little files with the various cases.

I’ve packed them and uploaded on the server. The idea is to create a test file for each problem I could find.

If you want to check them, you can download them here

As previously stated, I run tests over tests, to be sure I’ve not screwed anything. This time I want to share this pleasure with you: I’m posting the differences between the epubs, the .tex and the .html files.

epub diffs

TeX diffs

(If you notice large chunks of differences that seem not to have any sense, please let me know, a comment will do it)

The HTML diff is actually very small (just some more urls linkified )

Oh, I didn’t mention this. It seems that the core converters are ready and language-setting enabled. I’m still unsure if they should be installed right now (as they are full compatible replacements) or to wait. To put them in place seems the only way to truly test them, because the output should be checked by the uploaders (!, ok, just a dream).

I’ve already spent some words about the advantages, so I won’t repeat myself.

Today I’ve received from a buddy of mine a CD-ROM burnt from the latest torrent release:

Thanks!

Today I’ve finished to write the ConTeXt code to generate the various formats (a4, letter, a4 imposed and letter imposed). I’m still puzzled how much binding correction should I give them. The binding correction is the space reserved for the binding of the booklet, that must be taken away from the general page layout, because it will be “eaten” during the binding, i.e., you will not see it.

For now I’ve set it to 3mm. If you have a long arm stapler, you don’t need binding correction at all. On the other hand, if you have a good stapler, you can first fold the paper and then staple it from the outside. This is a bit lame. I’ve been doing it for a while, but it’s ugly and the booklet doesn’t read well (because you continue to struggle with the paper). Just for the record, I’ve bought a long arm stapler for 11.10€. It will be good enough.

The main problem is that the library got little or no feedback at all regarding the way the PDF are imposed. Anyway I have reasons to think that the PDF are being printed. Going a bit out of topic, I was thinking about buying an e-reader. Here in the banana republic that “gadget” costs about 250€ (for a 6 inches monitor). For my laser b/w printer I payed about 100€, if I recall correctly. With a single toner (about 50€) it prints out about 3000 pages. So, doing some math, it’s not really worth of. For 250€ I can print out a pile of things. And once is printed, the file comes really to live. It becomes real and can be shared further. If it remains a file on a drive, well, it’s just a long string of 1 and 0. So I’m not going to buy anything. Instead, I will print out. This was to say that this kind of feedback is not only appreciated, but needed to improve the quality of the archive.

Changing topic again, when I started the code to switch form XeLaTeX to ConTeXt, I was concerned about the performances. As you can imagine, producing pdfs is a very CPU intensive process. Today I had the first surprise: for a single output format, not imposed, the old script is faster, almost two time faster. But, if you consider the imposed ones too, the difference is really small. In some case the new system is faster.