Editorial Sustainability and Open Peer Review at Programming Historian
Humanities publications are often maintained by collaborators spanning vast geographic distances and uneven temporalities. Such logistical challenges are not trivial, as anyone who has grimaced at yet another version of a jointly-edited Word document arriving in their inbox can attest. These issues only multiply when creating or maintaining an online journal—especially one with a rolling submission process—constantly managing the collective work of authors, contributors, reviewers, and editors.
Programming Historian is an online, open access, peer reviewed suite of over 40 tutorials (and growing!) that help humanists (though slanted towards historians) gain experience and insight into not only programming, but a wide range of digital tools, methods, approaches, and workflows to facilitate their research. The success of the site is driven entirely by the energy and enthusiasm of volunteers (general editors, lesson authors, lesson reviewers, etc). Over the last year, we have adopted and refined an open and sustainable editorial and review process centered on Markdown and GitHub that may serve as a useful model for online journals.
Why We Needed A Change
As with many—if not all—volunteer projects, time is always the most limiting factor. Any workflow or publishing platform—no matter how popular or potentially useful—becomes detrimental when small tasks seem like they are taking longer than they should. It was precisely this issue that drove Programming Historian from WordPress to GitHub (and GitHub Pages).
WordPress was a nearly ideal solution for several years. However, as our publishing rate increased and our number of lessons grew, managing lessons (including finished lessons, lessons awaiting editing or reviewing attention, started but abandoned lessons, etc) became too unwieldy. Not only was it difficult to manage versions of lessons and the status of lessons at various stages of production, it was even harder within the framework of the WordPress administrative interface to manage the editorial process as we preferred. Surely we could have written better code when we made customizations over time, but such is the gap between ideal practice and practical reality within entirely volunteer projects.
What We Use
We first converted our existing lessons from HTML to Markdown (using the fantastically useful Pandoc conversion tool); we now require that our authors submit lessons in Markdown so that they can seamlessly enter our editorial workflow. Markdown is a plaintext syntax that allows for simple text formatting (headings, underline, bold, italic, lists, etc) without all the bulky tags and symbols that make HTML difficult to work with during the editorial process. The plaintext file format means that we don’t have to worry about shifting proprietary file formats, or bizarre formatting issues that inevitably arise when different people try to serially edit the same file in different word processing applications.
We now use GitHub to organize, manage, and version lessons, as well as other site pages and ancillary resources (images, data files, etc). GitHub is a platform used primarily by open source software developers that allows multiple people contributing to a particular project—working from a common code base—to develop sections of code independently without stepping on each others toes (namely, editing outdated versions of files). But GitHub’s utility goes far beyond keeping track of code. Its generic functionality to manage versions of files and their revision histories makes it ideal for managing content of an online journal, as well as facilitating open peer review of that content. While it can seem intimidating at first (especially compared to WordPress), excellent documentation makes the barrier to entry quite low.
Why We Use It
Moving to GitHub alleviated the burden of editing lessons (or other site pages) within an online administrative interface and eliminated the need to create separate HTML versions of the lessons. GitHub allows us to work with individual lesson files, offline if we choose, and easily (re)incorporate them into the Programming Historian website with minimal effort. Going from an easily readable file composed in Markdown to an HTML version of the lesson is effortless because all GitHub repositories (where the Markdown files are stored), come with built-in functionality called GitHub Pages that automatically takes pages written in Markdown (or HTML) and converts them behind the scenes (via Jekyll) into static HTML pages that comprise the working website. The built-in templating system is simple, yet highly functional (and designed for non-coders), minimizing our frustration getting our pages to look the way we want. Thus, the easily readable and editable format of our Markdown lessons essentially comprise the website content itself. This way, we have one canonical working version of every lesson, a feature that is essential for us not only during the editing process, but especially as editors and readers try to keep lessons up to date while versions of tools and technologies continue to evolve. There is always only one file in one place to update, and everyone has access to it via the open Programming Historian repository. If someone needs to see what a lesson looked like at a particular time (per traditional publication expectations), it is readily viewable via GitHub’s web interface.
The GitHub platform excels at providing the key functionality we need (an open, versioned repository), which makes it easier for us to focus on our lessons rather than spending time on the website infrastructure itself. GitHub’s many tutorials (like this one) quickly brings new authors and reviewers up to speed—and they can do everything they need to do via the GitHub website. Markdown is very simple to read and edit, making editorial work much easier than dealing with other file formats, whether HTML, Word, or PDF. The file system-like repository structure makes organization a snap. Versions of lessons in various states, which previously had been separate entities to manage, are now simply snapshots viewed in the GitHub history browser. Using GitHub’s built-in “issue tracker” (an “issue” is GitHub-speak for something that eventually needs to get done or resolved), any discussions about potential changes to the site are automatically archived for future consultation, which has been utterly invaluable for the editorial team.
Employing GitHub as a staging area for lessons-in-progress facilitates our commitment to an open peer review process. Authors who write new lessons contribute them to our repository (via pull requests) so that lessons are immediately online and accessible for editors and reviewers, but virtually invisible unless one knows about them. Editors then create an “issue” via the issue tracker; peer reviewers (once they’ve created a free GitHub account) attach their comments to it. As an open, public repository, all reviews are fully visible to the author and everyone else—and the reviewers get proper recognition and credit for the difficult review work. We believe that this transparency results in more open and honest communication, even amidst critical reviews. Our process thus embodies our philosophy that editors, authors, and reviewers can work together convivially to create significantly more useful resources than via secret editorial brokering and reviewer evaluation.
The community response to our move to GitHub was both productive and inspiring. We immediately received (and continue to receive) a continuous stream of suggestions for improvement (via issue tracking or pull requests) from folks who had no stake or involvement in a particular lesson, but have an idea for making it better. All changes are approved by an editor, but such suggested changes can be incorporated into the live site with the click of a virtual button (hence the power of using GitHub to manage live content). Productive discussions about technologies and techniques pertaining to the lessons have unfolded in the built-in commenting feature of GitHub, a fringe benefit that underscores the value of using an open, community platform for sharing knowledge, whether via formal lessons or informal comments and suggestions.
From a sustainability perspective, required resources are minimal: no hosting service to pay for, no platform installations to maintain, no special software required. We remain at the mercy of GitHub's stability, but so much is true for any hosting provider. The site remains entirely exportable and easily portable with a small footprint and few dependencies.
Skills and Resources
- Technical Skills: Basic concepts of version control, familiarity with GitHub’s functionality, editing plaintext files with a text editor, Markdown, HTML
- Competencies needed: Willingness to learn and adapt new workflows and processes, technical writing to produce documentation, familiarity with the terminal/command line is very helpful, but not essential
- Infrastructure needed: Free GitHub account
- Resources: GitHub, Markdown, Introduction to GitHub, Mastering Markdown, Forking a Repository, Using Pull Requests, PH Reviewer Guidelines, Reviewing Lessons with GitHub
1. How did your academic background, experience, and professional interests prepare you for this project, and projects like it generally?
In terms of specific academic training, not at all. But the ethos of the digital humanities community emphasizes diving in and learning new technical skills even when the immediate payoff is unclear. I've always loved that aspect of it, and that energy was a constant driving force to keep experimenting and learning, which makes it increasingly easy to pick up new technologies or processes. I suppose one has to have a general interest in learning about technology in the first place; some humanists seem to take pride in crafting an identity that's opposed to it, and that mindset is the most significant roadblock to a project like this, far more than anything technical.
2. If needed, what new things did you learn during this project (technical skills, managerial skills, team building, project management, etc.)? How did you approach learning them?
GitHub was familiar to most of the editorial staff, but not everyone. But no one really had much experience in terms of formulating a new editorial workflow, working it out with other editors, authors, and other contributors, as well creating necessary documentation. It was crucial that all the Programming Historian editors were able to communicate effectively about what was dangerous and exciting about adopting a totally new workflow centered on a new platform. This pushed some editors' technical skills forward; for others, it significantly enhanced their managerial skills. I think the entire editorial team was excited about trying to implement a new process that aligned with our values of openness and community building while trying to maintain an academic resource of the highest quality. It's easy to put in a lot of extra hours when they work directly towards something you feel strongly about and transcends the value of typical academic work.
3. Thinking on the project as a whole, ff you could distill your experience, what key lessons learned would you share?
Community participants and contributors like to experiment, too. We considered adopting our new workflows long ago, but we were very concerned that it would be too difficult for us as editors and our contributors to shift to a new process and might make us seem too inaccessible. But as soon as we had our new repository up and running, the community response was not only encouraging, but overwhelming. We had more engagement than we ever did. Even when our editorial policies were shifting as we made refinements and learned from experience what worked what didn't, our authors and reviewers were (seemingly) happy to follow the new protocol even though it wasn't always as smooth as it could have been. But far from being a deal-breaker for anyone (that we know about), it seems to have encouraged more people to get involved because it's a way of learning new technical concepts (if not skills) while making a scholarly contribution to their field.
4. Any final thoughts?
Don't be afraid to jump in and try new things. Programming Historian editors have monthly editorial Skype calls when we discuss how things are going, new directions we might want to go, and evaluate how well we're meeting our general goals with how we're doing things. We have spent months batting an idea back and forth, often overthinking how something should be implemented, or if it will really benefit us and what we're trying to do. A great idea could remain in limbo for months at a time; we'd forget about it and then have the same conversations a year later. Sound planning is important, but we've had the most success when someone (after productive brainstorming and discussion) simply takes the initiative to make changes, whether to establish provisional policy, redesign or rework a page on the site, or change our editorial workflow, etc. We know that nothing is ever un-doable, which gives us confidence to experiment. We communicate well, but not everyone can all be involved in all aspects of the site. When we allow each other to run with an idea, it usually works pretty well and everyone else eventually chimes in to improve the implementation. Even if it's not the most efficient change or even a mistake, we learn from that. Regardless, it's better than over-planning and under-executing.
The Programming Historian is an online, open access, peer reviewed suite of over 30 tutorials that help humanists (though slanted towards historians) learn a wide range of digital tools, techniques, and workflows to facilitate their research. Despite the name, we do not focus exclusively on programming, but rather aim to provide guidance on a variety of digital methods and approaches.