Check out our progress! This development status update chronicles progress on the Open Siddur made since our last update, February 15, 2010.

If you’d like to get news of Open Siddur Project development as it occurs, make sure to follow @opensiddur at Twitter, or join the opensiddur-announce email list. We also recommend following updates on opensiddur.org with our RSS feed. (Just visit this URL with your favorite RSS reader: http://opensiddur.org/feed/ .)

The creative work used in our traditional liturgies is the common cultural heritage of the Jewish people. Most of this work resides in the public domain. The Open Siddur is your Siddur. Join the Open Siddur Project today and begin crafting and sharing the siddur you’ve always wanted.

Project Overview

Ever wonder what our project looks like as a flowchart? Aharon updated the old one to be more readable — do you get it?

Did you want an Open Siddur web application last year and aren’t sure how you can help bring it into reality? Check out these ways you can help advance this project — even if you’re not a computer programmer!

Fascinated by technology and wondering how our work fits into the future of book publishing? Check out this link if you were wondering what a print-to-digital-to-print technology project such as our web application can offer the world more generally: http://thinkubator.ccsp.sfu.ca/wikis/xmlProduction/XMLProductionStartWithTheWeb

Project Team Updates

One way of helping to contribute to the project is offering work opportunities for our volunteers, some of whom are unemployed, freelance, and pay for their own health insurance. If you have a job opportunity and need committed workers and creative thinkers, contact us.

We welcome Shmueli Gonzales and Amir Starr Weg to our team of transcribers. Shmuel’s work can already be appreciated here.

Ben Varadi may have graduated Tulane’s Law School (congrats!) but the Tulane Center for Intellectual Property Law & Culture continues to provide the Open Siddur Project with excellent copyright research thanks to Justin A. Levy and an application that Ben devised, the Durationator. Ben’s also created a rather excellent Book Scanner. Ben’s now working for jgrad, a project of the NOLA Jewish federation providing Jewish resources for graduate students and recent college grads.

New Contributions

• A growing historical map charting Jewish liturgies by Aharon, now in version 2.3 with input from Dr. Richard Sarason (HUC-JIR) and Dr. Kay Shelemay (Harvard).
• A d’var tefillah and kavanah on the springtime Prayer for the Dew contributed by Rachel Barenblat.
• A kavanah on the meaning of one’s posture in the Amidah by Virginia Spatz.
• Two Simḥat Bat ceremonies were contributed, both from teachers at Yeshivat Hadar (Mechon Hadar): one by Rav Elie Kaunfer and Lisa Exler, and one by Dr. Devora Steinmetz and Rabbi David Silber.
• Twenty-seven translations of important halakhic source texts providing halakhic guidance on creative innovation with Jewish litury within the history of Rabbinic Jewish discourse. These were provided by Rav Ethan Tucker (Yeshivat Hadar, Mechon Hadar).
• A new English translation of Louis Ginzberg’s Prayer for the Government by Rabbi Tim Bernard.
• Modular transcriptions of the Nusaḥ Ha-Ari by Shmuel Gonzales. Nine modules so far. These transcriptions will be added to our transcription of Siddur Torah Ohr and proofread.
• A transcription of the 1917 JPS English translation of Shir HaShirim, the Song of Songs, was just completed by Aharon and proofread by Efraim.
• A digital JLPTEI XML formatted edition of James Strong’s venerable Dictionary of Biblical Hebrew, prepared with the help from Ze’ev Clementson, David Troidl, and Efraim Feinstein. Also, a digital JLPTEI XML formatted edition of the Singer Siddur, Rabbi Marcus Adler’s authorised siddur translated by Rabbi Simeon Singer. Both of these XML formatted editions are available for download with our source code.
• FONTS: Ze’ev helped convince the Culmus Ancient Hebrew Script Project to make their free (GPL) fonts even free-er with the GPL font exception. Google helped convince the maintainer of the Cardo font to share the font with SIL’s Open Font License.

Application Development  (step by step until it’s ready)

We tested and then made public our transliteration engine with eight transliteration tables to choose from including International Phonetic Alphabet, Modern Ashkenazi dialect, and Academy of the Hebrew Language.

Efraim documented and tested his encoding engine on the Singer Siddur.

Work on a replacement for our transcription/proofreading interface on our wiki is beginning. All transcribers are invited to provide input on the design and function of this transcription interface. (We want to make it easier for you to help us transcribe text.) Contact us directly or share your ideas on the opensiddur-talk discussion list.

Communications

We’ve made an effort to separate technical discussion into its own area for non-tech participants and followers of the Open Siddur Project.

Most recently we separated the Live Chat conferences we organize on IRC into separate tech and non-tech sessions. Logs are posted here.

Following his session at Limmud NY in January, Aharon Varady gave a public presentation on the Open Siddur Project at the Academy of Jewish Religon’s Spring Intensive in Riverdale, NY last March. (Other guest speakers included authors Lawrence Hoffman and Jill Hammer).

Aharon gave another public presentation on the value of Open Source for Jewish non-profits at the Future of Jewish Nonprofit Summit late last month in New York City. Afterwards, Aharon was interviewed by Radio613′s co-hosts Avi and Malcha, on CFRC Kingston, Ontario 101.9 FM.

Efraim presented the Open Siddur Project at the NewCAJE conference for Jewish educators. Follow-up thoughts are here.

New Documentation

Our Transcription Rules for transcribing and proofreading text were completely revamped.

Efraim wrote up an introduction to Hacking Open Siddur Code.

Many, many additions, disambiguations, and edits were made to existing pages on the wiki and at opensiddur.org .

To follow our development more regularly, follow our opensiddur RSS feed, twitter feed, and facebook group.

There is no single standard for Hebrew transliteration. In our demo you can transliterate Hebrew text in eight different ways originally set out in the following sources:

• Rules of Transcription from Hebrew Script to Latin Script (Academy of the Hebrew Language, 2007)
• International Phonetic Alphabet (2005, as used by Wikipedia)
• The SBL Handbook of Style (Society of Biblical Literature, 1999)
• Romanization Table for Hebrew and Yiddish (The American Library Association/Library of Congress, 1997)
• Concise Dictionary of the Words in the Hebrew Bible with their Renderings (James Strong, 1890)
• Coding for Transliteration of Hebrew (Michigan-Claremont, 1984)
• An approximation of Modern Israeli Hebrew pronunciation by Open Siddur lead developer, Efraim Feinstein (2010)
• An approximation of Modern Ashkenazi Hebrew pronunciation by Aharon Varady (2010)

Currently, the demonstration only provides romanization — the transliteration of Hebrew to a Latin script. By incorporating additional transliteration standards for additional scripts, we will be able to convert Hebrew to Greek, Cyrillic, Amharic, etc. (and vice versa). The tables are not fixed, and we can change them if bugs are found or better ways are suggested. Eventually, we will be implementing a table editor to allow editing the tables, creating, and of course, sharing new ones. For now, if you would like to add a transliteration standard to our database, take a look first at these examples.

The demo is linked to this post using an iframe tag; a direct link is here. The source code for this is open source, LGPL licensed, so you are free to take this and use it in your web application or website as well. Join us, and help make this a spectacular resource for everyone.

The form below provides a demonstration of this open source technology. Try it with some Hebrew now! If you don’t have any handy, try transliterating this phrase:

אֲדֹנָי שְׂפָתַי תִּפְתָּח וּפִי יַגִּיד תְּהִלָּתֶךָ׃

But dictionaries are not only invaluable reference tools for scholarly research. They are also useful features in online applications–and not only for their definitions. Dictionaries also include transliterations. Both translations and transliterations are features we would like to provide for users of the Open Siddur application we are developing. But in order to provide these features, the dictionaries must be digitized and their contents encoded in a standard searchable format.

Strong’s dictionary, prepared as a companion to his famous concordance, contains a complete list of Hebrew words that appear in the TaNaKh, transliterated with a consistent ruleset.  By formatting the words in the dictionary and replicating the transliteration as it appears in the dictionary the Open Siddur Project could test the transliteration engine that will be used to transliterate Hebrew text with nikkud (vowels) to any other script (Latin, Cyrillic, Arabic, Amharic, etc.).

Strong’s dictionary was digitized (PDF) in 1998 by unknown contributors. Converting the digitized, machine-readable text to an open standard format was a milestone sought by a number of projects including OpenScriptures, an open source project digitizing and encoding variant manuscripts of the Gospels. In partnership with Open Scripture contributing developers, the Open Siddur Project created a quality XML encoding of Strong’s dictionary. Work began early in February and was completed by the second week of April. The data and XML is available as public domain text, here.

The project served to test our transliteration engine and develop a good working and ecumenical relationship between two worthy open source projects sharing technology for advancing the digital humanities. The three main contributors on the project were David Troidl, Ze’ev Clementson, and Efraim Feinstein (lead developer of the Open Siddur). Ze’ev initiated the project in a forum discussion, here. (Ze’ev is a regular contributor to the Open Siddur Project and creator of innovative Jewish educational software for the iPhone/iPad platforms (so far)). Initially, David obtained Strong’s Hebrew data in the form of a PHP script from Dr. David Instone-Brewer. (The data is the basis for Instone-Brewer’s website 2 Letter Lookup).  Troidl then parsed out the data and converted it to the Open Scripture Information Standard (OSIS) XML schema, “using the best available OSIS structure for the data, since OSIS has no official dictionary module,” he explained. Efraim Ze’ev converted the XML to the Text Encoding Initiative (TEI) standard used by the Open Siddur Project. Efraim helped define the tag usage for the Open Siddur extension to the TEI — the JLPTEI.

James Strong is best known for his concordance and the scholarly tools he innovated show a prescient interest in linked data. Were Strong to look down from his perch in the heavenly yeshiva/academy at our work, I think he might be quite pleased with this collaboration. I asked David, Ze’ev and Efraim if they would comment on their work together contributing to the Open Siddur Project.

Why was updating Strong’s Hebrew Dictionary to Unicode and XML such an important target?

David: The existing ASCII transliterations were neither fully accurate, nor a faithful representation of the ones in Strong’s printed dictionary.

Ze’ev: Having a standard “dictionary” in Open Siddur allows us to provide new functionality that we didn’t have before. So, for example, someone might want to make a child’s siddur or a siddur for people who don’t know understand or read Hebrew well that contains definitions (perhaps at the bottom of the page or in the margin) for some of the less common Hebrew words. We now have the ability to add accurate transliterations (either alongside the Hebrew or instead of the Hebrew in the siddur) for people who can’t read Hebrew.

Efraim: It opened a route of collaboration between projects. Specific to the Open Siddur, working on this provided a specification for a new type of associated data — dictionary words.

How did this collaboration advance your project’s specific goals?

Efraim: By having a second independent implementation of transliteration we could debug our transliterator and discover corrections to make in Strong’s original document.

Ze’ev: The collaboration with David was great in that it helped us to identify a number of shortfalls/bugs in the existing Open Siddur transliteration code. However, the collaboration worked both ways in that, as a result of the email discussions, David was able to make changes to his source document (e.g. – adding qamets qatan, holam haser for vav, spelling corrections, etc) and Open Siddur was able to “robustify” the transliteration code so that it handled more “corner cases” than it did before (e.g. – adding dagesh transliteration support for non-bgdkft letters, improved handling of silent letters, support for literally-transliterated tetragrammaton, etc) and it also helped us to generally test/verify other transliterations. We also now have a “transliteration tester” which lets us automatically validate our programmatically-generated transliterations against static versions of the Strong’s transliterations (this can be used in unit tests in the future to ensure that transliteration code changes don’t “break” the existing Strong’s transliteration logic).

David: Ze’ev’s questions about Strong’s transliterations, while developing his transliteration table, spurred me on to work on a project that I had been wanting to do for some time anyway.  After using the existing ASCII transliterations as a comparison metric, we started trading our results, to compare the Unicode transliterations.  This allowed both of us to fine tune our code and produced a significantly higher degree of accuracy in the transliterations themselves.

The XML and code generated by this effort are available to everyone with open/free culture licensing. Besides this obvious advantage, how will this work help other folk’s projects worldwide?

David: Many Internet resources that I’ve come across fall far short of a print publisher’s tolerance for error.  While the availability of the information is commendable, I would like to see a greater emphasis on the accuracy of existing resources.  There are many texts, in print and electronic, that utilize Strong’s numbering system, and a reliable reference will be a benefit.

Ze’ev: The major additional “public” benefit from the exercise is that there is now an open-source, digital Strong’s source document that has accurate transliterations in the style defined in the Strong’s Concordance book and which has been edited to correct misspellings (both Hebrew misspellings and errors in the English transliterations). Already, at least one application (my Hebrew Bible iPhone/iPad app) is utilizing the Strong’s Concordance data and currently provides integrated word definition display when reading the TaNaKh  as well as Hebrew root lookup.

What new opportunities for collaboration and development do you see coming out of this work?

Efraim: One thing we (Open Siddur) don’t have is a way for the general public to get access to it!  It’s in SVN, and we have a way to convert it to HTML, but no public interface [exists yet] to transform it or to look up words.

David: The ideal markup format is still an open question.  My version is pushing the limits of OSIS, in its present form.  The TEI form that Ze’ev [Open Siddur] is using is verbose and repetitive.  I have some ideas that are still coalescing.  Also, we are moving toward adding a Brown, Driver, Briggs layer on top of the Strong’s data, striving for a richer and more accurate dictionary.

Ze’ev: This was a bit of a unique scenario in that we were both attempting to validate independently-developed transliteration code with the same source document. We have now completed the validation of the Strong’s transliteration scheme and have discussed following through with a similar exercise for the SBL transliterations.

Fonts on computers represent the intersection of graphic design and computer programming. This is because digital font files not only contain the typographic designs for letters, they also contain the logic for where diacritical marks (vowels, cantillation marks, etc.) appear in respect to machine readable text. In this way a digital font of an aleph is distinguished from a digital image of an aleph.

Being committed to both open source and collaborative publishing, the Open Siddur Project has been exclusively using Hebrew fonts distributed with the SIL Open Font License. SIL fonts are truly open source fonts which can be freely modified and redistributed and is thus compatible with the other licenses we are using. Imagine a siddur where different liturgical sources are displayed with different fonts and colors indicating their period of authorship! At the very least, a diversity of fonts provides choice for our users.

Although there are very nice Hebrew fonts available for a fee (Koren), and some even for no cost (Society for Biblical Literature, SBL fonts), these fonts aren’t free or open in their terms of use. Most Hebrew fonts are thus incompatible for use with our Open Source project. We are thus always on the lookout for fonts and font designers that are sharing their work with free and open software licenses.

Enter the Culmus Project! Our project was specifically interested in Yoram Gnat’s Ancient Semitic Scripts Fonts since these fonts use the same logic as SIL’s Ezra SIL font for the placement of diacritical marks in Hebrew. (This logic is licensed with an MIT license.) Even sweeter, the fonts are attractive and the two “Keter” fonts would make a nice alternative to Ezra SIL for Open Siddur users.

According to their website, Culmus “aims at providing the Hebrew-speaking GNU/Linux and Unix community with a basic collection of Hebrew fonts for X Windows.” While X Windows offers a graphic user interface for running window managers in the GNU/Linux operating system, all the software distributed with the operating system — including the fonts — must also be free under compatible licenses.

There was only one problem. The fonts were licensed under the GPL (General Public License), a widely used free software license incompatible with the aims of our Open Siddur. Due to the language of the license, fonts could only be embedded in GPL licensed documents. Embedding GPL licensed fonts in distributed non-GPL documents (such as PDFs) was forbidden. Considering the Open Siddur Project intends to provide users with their custom siddurim in PDF and other digital formats, software distributed under the terms of the GPL was off-limits.

Thankfully, because this problem had been longstanding within the Open Source community of projects, a workaround was available — a so-called “font exception” for works licensed under the GPL. Still, the Culmus Project hadn’t distributed its fonts with the “font exception” clause.

Noting these issues on the Open Siddur listserve, Ze’ev Clementson took the initiative to contact Yoram Gnat and see whether the font exception could be added to a future version of his fonts. The good news first came a few weeks ago when Ze’ev reported that Yoram had agreed to add the exception. And today we are so very pleased to report that the latest version of Yoram’s fonts are available for download and include the compatible language for use with the Open Siddur Project.

Congratulations to Yoram Gnat and Ze’ev Clementson for your kind work, and for sharing it with the rest of the world. May your intentions be preserved and inspire us.

Friends,

The communal project of Jewish spirituality can only be improved through cooperation and collaboration. The creative work used in our traditional liturgies is the common cultural heritage of the Jewish people. Most of this work resides in the public domain. The Open Siddur is your Siddur. Join the Open Siddur Project today and begin crafting and sharing the siddur you’ve always wanted.

This development status update chronicles progress on the Open Siddur made since our last update 11/11/2009. If you’d like to get news of Open Siddur Project development as it occurs, make sure to follow @opensiddur at Twitter, or join the opensiddur-announce email list.

Contributions (Aharon, Anonymous, Gabriel, Efraim, Eve, Daniel, John)

Software Development (Efraim, Ze’ev, Ilan, Raphael)

Documentation (Efraim, Aharon, Ze’ev)

Organizational Structure (Aharon, Efraim)

Communications and Promotion (Aharon)

Press (Aharon, Efraim)

Team Member Updates (Azriel, Aharon)

Wishing you a happy and warm Adar,
Aharon Varady
Founder & Co-director
The Open Siddur Project
http://opensiddur.org/contribute/join-us/

Our third development status covers progress on the Open Siddur made since our last update 9/22/09. Email aharon@opensiddur.org if you want to include something we haven’t covered. For now we’ll be sending these out once a month but if you’d like to get news of Open Siddur as it happens, make sure to follow @opensiddur at Twitter.

Contributions

Aharon read through Reb Zalman’s weekday and shabbat evening siddur and prepared the document for sharing as a PDF as well as in the ODT (open document) format. Folks can now use this to help make their own siddurim offline while we continue to work on creating our online open siddur web application. Want to craft your own siddur using material from Reb Zalman’s siddur? Check it out here.

We need a *print* copy of Siddur Torah Ohr (pre-1923) to correct and proof a digital transcription of the siddur already available CC-BY-SA at wikisource. Do you have one we could use? Please let us know. With this text vetted and proofed, we could use it to help folk begin making their own siddurim as well as for comparing it with our transcription of Siddur Avodat Israel.

Manual transcription of Seligman Baer’s Siddur Avodat Israel slowed over the last few months, but this is the easiest place someone with little technical capability can make a big difference. Every line of text transcribed is digitally liberated for use in future siddurim. If you haven’t yet, register on the wiki and start transcribing here.

Software Development

Efraim completed an implementation of XPointer to be used in our XSLT transforms and XQuery.  In the process, he developed a generic parser generator for XSLT (http://wiki.jewishliturgy.org/XSLT_Grammar_Parser).  This is a prerequisite to the continued development of (1) toolkit APIs that grab ordered text segments from JLPTEI documents and (2) transforms to convert JLPTEI to other forms. The code is complete and committed to subversion.  The code has had some minimal testing.
Efraim explains in more detail in a posting on our listserve: http://groups.google.com/group/jewishliturgy-discuss/browse_thread/thread/61285fef05e1846f

Efraim also finished a first pass of XSLT code to combine multiple overlapping hierarchies as described on our wiki.  Finishing this take us one step closer to processing JLPTEI.  Help wanted!

Azriel’s OSNAT (Open Siddur Network Application for Transcription) is still pre-alpha, but includes a fly in-browser Hebrew Unicode 5.0 standard keyboard. Interested and familiar with Javascript and jQuery?

We are always looking for more software developers! Please contact us with your skills.

Documentation

We are still looking for volunteers to just look over our documentation and help us know how it reads and where we can make improvements. Anyone can freely register to edit on our wiki.

Organization/Structure

w00t! The Open Siddur now has a mission statement. see http://wiki.jewishliturgy.org/Mission_Statement

How can we better track progress now that we have a list of milestones (http://wiki.jewishliturgy.org/Milestones)? Aharon is thinking of a fractal tree showing milestone markers and child dependencies.

Communication and Promotion

We had our third Open Siddur Open Chat at irc://irc.freenode.net/jewisliturgy on October 16th. The logs of the chat are available on our wiki at http://wiki.jewishliturgy.org/IRC_Conference

Our next Open Chat is scheduled for November 22nd, 11am EST.

Team Member Updates

At Yeshivat Hadar, Aharon is studying with R. Elie Kaunfer the evolution of Jewish spiritual practice in prayer and meditation and where Jewish liturgy supports this practice in the evolution of nusaḥ.

In actuality, there is no such thing as a single text called the Ashkenazic siddur, the Hasidic siddur, or the Sephardic siddur, etc.  A nusaḥ is a major division which uniquely specifies a common denominator of customs within a group of customs.  Within Ashkenaz, there are differences between the Polish and German customs.  The Iraqi custom is not the same as the Yemenite custom, and the Lubavitch custom is not the same as the Breslov custom.  There are also divisions within each rite along major the philosophical boundaries that have developed in recent centuries, which lead to differences in custom and text.  The traditional-egalitarian rite (usually a variant of the Ashkenazic rite), for example, is still undergoing major evolution.

Now, let’s say that we provided a single text for each sub-rite within the major rites, and simply copy-pasted the common text between them.  The amount of replication of very similar material between the texts would be huge.  If a mistake were found in one copy, it would have to be corrected separately in all copies.  Further, there would be no real connection between the copies other than their content.  If we were tracking some data (say, grammatical or historical data about the text), it too would have to have each change copied to all copies of the texts.  This would quickly result in an unmaintainable mess.

The Open Siddur’s current approach is different.  It involves (1) minimizing the amount of stored text, (2) storing the differences between the texts and (3) sets of conditions that specify when each variant is selected.  If a typo is corrected in one variant, it is corrected for all variants.  Any stored metadata is also automatically consistent between all texts.  An additional advantage of this approach is that a community with a custom that differs from the “base” custom of the rite only has to make a different choice of variants.  No change is required in the text in order to support a slightly differing custom.

The disadvantage of this approach is that each user likely only wants one variant of the text.  Something has to (1) convert human-typed text into the unified format used in the archive and (2) splice the archival text into a unified “printable” text that can be used as a siddur.  And that is one answer to why we need to develop software.

Our second development status covers progress on the Open Siddur made since our last update 8/23/09. Email aharon@opensiddur.org if you want to include something we haven’t covered. For now we’ll be sending these out once a month but if you’d like to get news of Open Siddur as it happens, make sure to follow @opensiddur at Twitter.
Contributions

A beautiful scan of the Routledge Mahzor was contributed to the Internet Archive by the Princeton Theological Seminary Library (see http://groups.google.com/group/jewishliturgy-discuss/t/433a218e6f5a130b). Other important scans of siddurim in the public domain have been uploaded by the University of Toronto and the YIVO Institute. Scans contributed through the Internet Archive provide the Open Siddur project with a well of texts for digital transcription.

We are looking for a copy of Siddur Torah Ohr to correct and proof a digital transcription of the siddur already available CC-BY-SA at wikisource. Do you have one we could use? Please let us know.

The Tulane Center for Intellectual Property Law and Culture is consulting with Open Siddur on the copyright status of extant works that may be in the public domain. Thanks Ben and Justin!

Documentation

How can folk contribute material to the Open Siddur Project *right now*? See our new http://wiki.jewishliturgy.org/Submissions_HOWTO for more information.

We are looking for volunteers to just look over our documentation and help us know how it reads and where we can make improvements. Anyone can freely register to edit on our wiki.

The Jewish Day School systems are filled with many talented students, and we would like to reach out to those interested in technology and computer science, particularly in high schools.  The Open Siddur Project offers an opportunity for these students to participate in a real free software and open source project that is also relevant to the entire Jewish community. We invite them to participate in development discussions, provide original research, learn and write programs in a number of computer languages, edit, proofread, and correct texts, and work with others of many different backgrounds, in a pluralistic Jewish context.

We will be happy to recognize the participation of your school or project represented by its volunteer programmers. To participate, please invite your students to fill out our survey form available at http://opensiddur.org/contribute/join-us/.* To browse our websites please visit, http://opensiddur.org and http://wiki.jewishliturgy.org. To join our discussion list directly, please visit http://groups.google.com/group/opensiddur-talk/. Please feel free to contact us via our contact form or via email for more information.

(*) While we are not a commercial entity, in case we ever do fall under the Children’s Online Privacy Protection Act of 1998, we avoid collecting personal information from students under the age of 13.