In which I describe a process where we start on Overleaf and set up an academic blog on Medium with references and all; you cannot format references in Medium to write academic papers. Here is one way to do it.
The web and online collaborative writing and version control systems are influencing the process of scholarly communications. Jaret Byrnes (Byrnes et al. 2013) wrote in 2013 that the future of scholarly publications will rest on four pillars: an ecosystem of scholarly products, immediate and open access, open peer review, and full recognition for participating in the process. Reflecting on version control and its influence on academic writing, Karthik Ram (Ram 2013) wrote a tutorial on Git and version control integral to reproducible scholarship and in maintaining codes and artefacts of knowledge. An increasing number of Resbaz events are organised all over the world that enable emerging and established researchers in using digital tools. Writing in “Opening Science”, Cornelius Puschmann (2013) wrote
“Civil society expects to be engaged in a dialog with science, rather than being lectured. The affordances of social media (blogs, wikis, social networking sites) should accordingly be regarded as supporting a general long-term shift towards a more egalitarian relationship between experts and the lay public, rather than driving it.” (Puschmann 2014)
Puschmann’s argument suggests that there is a need for integrating the more formal style of scholarly writing with the less formal engagement that happens in social media. This “hybridisation” is not easy, yet this is needed in an age of increasing influence of social media. Recently, long-read formats of Medium has become popular in presenting diverse viewpoints. Yet Medium lacks a formal structure that can allow more academic or scholarly arguments to be presented here. For instance, using references and citation based attributions, tables of data; on the other hand, it provides an excellent environment to read and “fork” ideas across the board and brings on board a wide range of readership to share and spread ideas. The challenge then, is to connect the world that more formal scholarly communication provides in terms of authentic and vetted information and the large, dynamic readership that can take that information and spread that Medium affords.
Here, I’d like to write a workflow where we can bring these two processes together. I am going to write about a few online tools that together can meet the needs of four pillars that Byrnes wrote about: creating an ecosystem of scholarly products, making access immediate and open, allow for open peer review, and full recognition for participating in the process. First in this set of tools are the openly accessible collaborative online Latex or Markdown based tools used for academic writing. Latex is a well-established tool for academic writing, and with online ‘web skins‘ that allow for rich text interface based writing, it has become easy to write and collaborate. I use Overleaf on a daily basis and it meets all my writing needs. There are other tools as well, for instance ShareLatex, and Authorea. All of these allow for easy writing interfaces and a user can get a free account on these services and can write documents both in private and as public documents and share the documents online so that his or her collaborator can write using a web interface. Besides LateX for writing scholarly documentation that is also web readable, Markdown is another format that is widely used for web authoring. John Gruber authored Markdown. Markdown not only makes authoring web based documents easy, but there are other advantages over LaTeX in being easily readable as well with simpler set of markup codes. However, native Markdown does not support for insertion of citations and tables for instance, and hence several authors extended Markdown to author more complex documents. Here is a side by side comparison among markdown, latex, and html.
As you can see in markdown, the codes in the text are more readable than either in raw html file or raw latex file. Therefore, both latex and html files need to be rendered before they can be so-called ‘human readable’, whereas with markdown, they can be readable even when they are presented on a raw text file format. Besides, in markdown, html codes are admissible. The original markdown format underwent expansion: Fletcher Penney wrote Multimarkdown package that extended the functionality of Markdown to allow for tables, and citations. Another variant of Markdown include the Github flavoured Markdown, which is used in authoring text in github.
Markdown syntaxes provide an excellent set of easily applicable rules that you can use to write in plain text using nothing more than a plain text editor such as Notepad (Windows), or Textedit (Mac) or indeed any plain text editor. You can also probably use a word processor programme to author Markdown documents but these are best avoided. This makes Markdown both ubiquitous and useful at the same time, being platform independent. Yet, for many authors, document conversion from one format to another is important. This is easily achieved with John Macfarlane’s Pandoc, often referred to as the Swiss Army Knife of the document conversion. Recently, scholarly markdown project by Karthik Ram et al (2012) have expanded the scope of markdown to write scholarly papers. The work is incomplete, but provides an excellent scope of development in this field.
Pandoc and scholdoc are ways to convert documents written in plain text into readable units that are also compatible with other software programmes such as a web browser or a word processing software. However, for collaborative writing and transparent research, one also needs a tight version control system so that multiple versions of the document can be reviewed for changes and reverted or worked on. Git provides this functionality. The online git book provides a good overview of how version control with git works. Essentially, one downloads git to the computer on a folder, and then connects with an online repository as a remote machine and exchanges files. The files are staged or added and then committed when changes are made and pushed and pulled from the distant repository. The github is an online git repository that anyone can use for maintaining version control and get for free to store and use files for up to one GB size. Here is a freely downloadable git cheat sheet .
Now we have a way of writing plain text documents, some intuitive (markdown) and immediately machine and human readable, and some predominantly machine readable (html and latex), this is the time we put the document up to the web. Hugo is a Go programming language based site generator. It is fast and easy to learn. You can download the Hugo programme from the following site https://github.com/spf13/hugo/releases
After you have downloaded the sources, and installed Hugo, you create a new folder for your website, and within the folder set up a Hugo site with the following code:
hugo new site sitename
Hugo will create a new site with the following files and folders:
For this workflow, we will work on the folders content, themes, static and will edit the config.toml. Content will contain your files (posts, articles, etc), static will contain images and I usually also keep the bibliography files (bibtex files that Pandoc will use). Create an img folder within the static folder from the parent directory:
Themes will initially be empty and you will need to populate the themes with a specific theme. You can find themes in the Hugo themes website and here, we shall install a minimalist Hugo theme from the following website: https://github.com/digitalcraftsman/hugo-minimalist-theme
Use the terminal, and from the parent directory, write the following code to install the hugo-minimalist-theme:
git clone https://github.com/digitalcraftsman/hugo-minimalist-theme.git
After installation of the hugo-minimalist-theme, if you navigate down to the exampleSite of the hugo-minimalist-theme folder within the themes folder, you will see a config.toml file. Copy that file to your parent directory. Also copy the two images you will see in the static/img folder within the themes folder to your static/img folder of your parent directory.
So, here’s the code (from the parent directory):
cd themes/hugo-minimalist-theme/exampleSite (Navigate to the example Site)
cp config.toml /Documents/demo/ (will copy the config.toml to the parent folder)
cp static/img/*png static/img/*jpg /Documents/demo/static/ (this will copy the images within the img folder to your static/img folder of the parent site
Finally, open the config.toml site in a text editor and configure:
Must fill in the base url, in my case, I call it arins-papers.surge.sh (you can give it another name); note the format, x-y.surge.sh even though we have not configured surge yet we will do that soon.
In the next step create a post on Hugo by using the following code (I shall call the file hello-world.md):
hugo new post/hello-world.md
Note well that this is a new post that will be created in the content folder of Hugo website and it is a markdown file. The beauty of a markdown file is that, you can weave html and markdown codes in it, so make sure it is markdown. Also, make sure you are in the parent directory and that it is post/filename. That forward slash is important. This will create a new file named hello-world.md in the folder content/post (it will also create a new folder titled post btw).
If you then edit hello-world.md, you will see something like this:
But you will see that this is minimalist and so we will change the top half of the document between three plus signs (this is known as TOML style meta data). So we change the title to ‘Hello World’, and added some tags such as ‘tutorial’, etc. We still keep the body of the hello-world.md empty. For now we can fill with some simple information, such as second level header in markdown and some dummy text. We will fill this page later with information from Overleaf.
So, here’s what you do:
- Build the website with hugo first, so do
hugoat the command prompt. You can optionally indicate the theme. You must be in the parent folder to do this, otherwie it won’t work.
- This will create a folder titled ‘public’ that will flatten your site
- Now push it to surge with
surge public/ your-site.surge.sh
In our case, let’s say we create a demo site with:
hugo ; surge public/ arins-papers.surge.sh
At the end of these steps, here is a screen-shot of the site:
Nothing fancy. Very barebones at this stage. This will change.
If this is the first time, Surge will ask for your token and email address, and after the first time, you can just issue the following code to deploy:
rm -rf public/ ; hugo ; surge public/ your-site.surge.sh
Remember that every time you change your files on the site and need to build the Hugo site, remove or delete the public/ folder and then let hugo rebuild it from scratch.
Connect your hugo site with Overleaf
Now we are in the home stretch. First, get a free Overleaf account. Then, when you log in to Overleaf, you will see you can start a new project. Start a blank Overleaf document. Then, visit the “SHARE” button on the top of the Overleaf editor and note the clone with git url (see figure):
I assume that you have pandoc and git already installed in your computer. You can test them by:
git –version (will return a valid value)
pandoc –version (will return some information)
Step by Step setting up the connection between your hugo blog and Overleaf
- from the parent directory, create a git repository
- connect to Overleaf
git remote add overleaf URL(URL is your unique Overleaf url)
- add the content/ and the static/ folders nothing else
git add content/ static/
- commit the addition and push
git commit -m some message
git push overleaf master
You will see that Overleaf will populate with content/ and static/ folders in the folder space. Add a new file and name it something like ‘article.tex’ in the post folders. Write in Overleaf. Add a bibliography file and call it ‘refs.bib’. Populate the refs.bib file with bibliography information.
When you work on the Overleaf side, keep in mind that:
- Put all bib files to the static folders
- Put all image files to the img folder within the the static folder
- When adding images in Overleaf, use static/img/filename.extension
- Keep the PDF generation and the latex code generation process separate
Pull the Overleaf article to your git repo and Pandoc it
Now, from your parent folder, pull in the files and folders within your Overleaf repository. To do that, from within the parent directory, do:
git pull overleaf master
This will pull the overleaf folders and files.
Then use pandoc to convert the latex files to html and concatenate the contents to the hello-world.md file:
pandoc -f latex -t html content/post/article.tex –filter pandoc-citeproc –bibliography static/refs.bib | cat >> content/post/hello-world.md
Now modify the hello-world.md in a text editor and edit. Essentially,
- From the image tags in the Overleaf document, remove ‘static/’
- Add ‘##References’ at the end of the document before the reference information
From within the parent folder use hugo to build the site:
rm -rf public/; hugo; surge public/ arins-papers.surge.sh
Now open Medium in a compatible web browser that is not a mobile browser. Click on ‘Stories’, then ‘Import Story’, and fill in the surge website URL. Follow the instructions and your page will be imported to Medium.
Byrnes, Jarrett EK, Edward Baskerville, Bruce Caron, Cameron Neylon, Carol Tenopir, Mark Schildhauer, Amber Budden, Lonnie Aarssen, and Christopher J Lortie. 2013. “The Four Pillars of Scholarly Publishing: The Future and a Foundation.” PeerJ PrePrints.
Puschmann, Cornelius. 2014. “(Micro) Blogging Science? Notes on Potentials and Constraints of New Forms of Scholarly Communication.” In Opening Science, 89–106. Springer.
Ram, Karthik. 2013. “Git Can Facilitate Greater Reproducibility and Increased Transparency in Science.” Source Code for Biology and Medicine 8 (1). BioMed Central: 7.