Quarto describes itself as “an open-source scientific and technical publishing system”. With the work that I have been doing in the lab, I’ve been hoping to build a “future-proof” method of writing documentation, visualizing results, and perhaps even writing abstracts / papers, in a method that can easily be picked up by other people with varying technical backgrounds.
LaTex is frequently considered the ‘gold standard’ in the physics, math, and engineering worlds. However, to write even a single paragraph document, you need to have a bunch of preamble code, opening / closing statements, etc. Markdown is wonderful in that a person can easily read the raw markdown code or see it rendered to HTML easily on sites like GitHub or on a WikiJS. Jupyter Notebooks allow for markdown to be present alongside code, but is overall intended to be a ‘notebook’. I was looking for something like markdown, but with a little more capability, and the ability to add in LaTex (for equations and formulas) as needed. This is when I discovered R Markdown. The solution seemed perfect except for one thing – besides knowing that “in R-studio, you can generate a graph with ggplot”, I pretty much knew nothing about R, nor did I want to be restricted to using an R compiler. In comes Quarto!
The Quarto use-case
(But first, why I’m writing this)
To be upfront with this post, my initial motivation was to document what I am doing in case I forget how to reproduce my own work for myself! Though I can’t find the link for it at the moment, I believe it was Matt Might, a professor of Internal Medicine and Computer Science at the University of Alabama at Birmingham, who described in his blog how he approaches responding to questions from students
- Reply (to the student only)
- Reply all (to the whole class)
- Reply to public (when a similar question may be asked frequently or the answer could be beneficial for more than a small group, you can post your answer online, publicly)
In an effort to write more frequently and get past my imposter syndrome, I’m hoping to combine my need to document with Dr. Might’s concept of ‘Reply-to-Public’ in this post and my Instances “blog”.
For the last 1.5 years, I have been working as a Post-Doctoral Research Fellow at Rush University in Chicago. During this time, I have learned a lot from my peers and mentors on bench lab techniques and approaching various scientific questions. Additionally, I have come across a number of new tools or ‘created’ new approaches for making my research more efficient, documented, machine-readable, and reproducible. My ‘sum knowledge’ is spread across my lab notebooks, Datalad/Git repositories, and even scratch paper. Following the path which arXiv brought to bioXiv, I want to open parts of my lab notebook. The selfish reason first: I have had to look up the protocol for making a 10% Formalin (4% PFA) solution in PBS, after running out of my 10x PBS stock, an embarrassingly significant number of times. And I would prefer not having to log into LabArchives on a laptop or deal with its buggy phone app, especially while working with toxic chemicals. Along the way, if it helps others in my group, institution, or researchers anywhere, a ‘Reply to Public’ post may be worthwhile. It parallels one of my favorite aspects of the internet: FOAMed, or Free Open-Access Medical Education.
Additionally, there are many differing, often opposing, protocols for common experimental techniques (e.g. Western Blot) and uncommon ones (e.g. RNA isolation from human articular cartilage explants – a particular challenge I faced in the early months of my postdoc). By opening up my approaches and rationale, an informal peer-review can begin at an earlier stage. ResearchGate is great in leveraging a community, but has inherent limitations in the depth of question or conversational capacity users can have. Medical researchers can learn from their computer science counterparts, who track changes with version control, raise bugs as “issues” or ideas as “enhancements”, and incorporate frequent, incremental changes with “pull requests” into the main (historically called ‘master‘) version that everyone is using. In open-source software, this is all publicly available and anyone can contribute; the most popular host for doing so is GitHub, where you can also find my public repositories (@pranavmishra90).
Initial test-cases with Quarto
My first trial with Quarto will be changing my small, but growing, markdown formatted research-reference repository at GitHub. This will be the starting place (as a Datalad supradataset) which will include documentation for other repositories, pulled in as git submodules (datalad subdatasets).
If you do not know or understand any of the technical terms listed above or code shown below, it is okay! Quarto will create a simple website for you to navigate through just like any other website.
Pre-requisites
Python
Mamba is similar to the ubiquitous Conda from Anaconda. It has major advantages in the speed it takes to detect your current environment and decide which version of packages to install. Any person who has used even medium sized conda environments knows what its like waiting for conda to determine what to install.
# Install mamba from Mambaforge (automatically includes conda-forge) curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh" bash Mambaforge-$(uname)-$(uname -m).sh
Quarto
#Note, update your code as quarto publishes newer releases wget https://github.com/quarto-dev/quarto-cli/releases/download/v1.3.333/quarto-1.3.333-linux-amd64.deb sudo dpkg -i quarto-1.3.333-linux-amd64.deb
Jupyter
Add the following to your Jupyter configuration file, which is commonly located at ~/.jupyter/config.json
{
"CondaKernelSpecManager": {
"kernelspec_path" : "--user"
}
}
Then, you will need to start a Jupyter session, which you can do by opening a Jupyter Notebook or Jupyter-Lab session. The python kernel from this session can be passed into Quarto
# Start a jupyter notebook in the current directory jupyter notebook . # Determine python kernels available jupyter kernelspec list
Output:
Available kernels:
python3 /home/pranav/.local/share/jupyter/kernels/python3
Take this output and include it in the YAML block at the top of an R-markdown (.rmd) file. In the VSCode settings, you can also configure the default kernel to run when executing code that requires python.
Hello World
We can test quarto in VSCode by creating a hello-world.qmd file.
---
title: "Quarto Basics"
format:
html:
code-fold: true
jupyter: python3
---
For a demonstration of a line plot on a polar axis, see @fig-polar.
```{python}
#| label: fig-polar
#| fig-cap: "A line plot on a polar axis"
import numpy as np
import matplotlib.pyplot as plt
r = np.arange(0, 2, 0.01)
theta = 2 * np.pi * r
fig, ax = plt.subplots(
subplot_kw = {'projection': 'polar'}
)
ax.plot(theta, r)
ax.set_rticks([0.5, 1, 1.5, 2])
ax.grid(True)
plt.show()
```
Output
Great, now you’ve rendered your first Quatro Markdown file!
Using Quatro to build a site
It is easy to create a ‘site’ project using the VSCode plugin. If you prefer the command line, you can create a new project with:
# Initialize a new site quarto create-project mysite --type website # Preview the site quarto preview
My _quarto.yml file
While you should tailor your _quarto.yml file to your own needs, if you need a starting place, here is the file I’m using on one of my quarto projects. The file creates a quarto website project with the following features:
- Preview on http://localhost:3333. While you work, any files that you edit and save will be automatically re-rendered (with a 10 min timeout)
- Save the render website to a folder called
_code/ - Create a left side bar with an automatically generated tree of links.
- A top navbar is also created with static links
- Links to the GitHub repository with on-page links to edit or raise issues
- A website footer
project:
type: website
output-dir: _site
render:
- "*.qmd"
preview:
port: 3333
browser: true
watch-inputs: true
timeout: 600
website:
title: "Research Reference"
navbar:
left:
- href: index.qmd
text: Home
- href: about.qmd
text: About
search: true
open-graph: true
sidebar:
style: "docked"
contents: auto
page-footer: "Created by Pranav Kumar Mishra || License CC-BY-SA"
reader-mode: false
repo-url: https://github.com/pranavmishra90/research-reference
repo-actions: [issue, edit]
format:
html:
theme:
light: flatly
dark: darkly
toc: true
lang: en-US
email-obfuscation: javascript
code-fold: true
grid:
sidebar-width: 350px
execute:
freeze: auto
cache: true
Bibliography
You can choose from a few methods for citing your sources:
- Included .bib file with BibLaTex citations (among other formats)
- Zotero Web API (and connectivity to desktop)
- DOI, Crossref, etc lookup
The “Visual” editor mode is the easiest way to confirm that the citation you wish to enter is valid and will be picked up by the pandoc engine which quarto utilizes. In VSCode, you can right click and select “Edit in Visual Mode”
You can easily add a citation by pressing Ctrl+/ and searching for “Citation”. The window will show you a list of references included in your source list (which in my case is often a bibliography.bib file).
