Converting Inaccessible STEM Content to Accessible Web Content

---

Inclusive Publishing Conference, NFB, 1/10/2020

Motivation

• STEM content moves online all over higher education
• Covid-19 is a serious driver behind this
• I am concerned about ephemeral publishing
  • Every day learning material
  • Lecture notes and handouts
  • Content in LMS, forums, blogs, etc.
  • Academic publishing, research papers, conference proceedings
• When I say STEM, I will be talking about Math, Statistics, Chemistry

Problems

• Poor education about what exists and is possible
• “PDF automatically makes content accessible”
• “Our officers want JPEG graphics; they don't know what to do with SVG.”
• “My level of mathematics cannot be made accessible.”
• “LaTeX documents are for print only”
• “They want me to start using Word”

Learning Outcomes

• Poor information leads to poor design decisions
• Work with authors and the information they can provide
• Do not impede their creativity
• Covid-19 forces a lot of content online. If done badly we might have lost a valuable opportunity
• Conversion is important
• But never through away information

Aims

• Pragmatically making STEM content accessible on the web
• The technologies and components are already there
• The web is syntax, the semantic has to come from elsewhere
  • Note: LaTeX, MathML, CML, MOL, etc. are all only syntax!
• Web technology can embed and hide the semantic
• Suitable semantically rich structure allows for
  • Accessibility
  • Support of users with diverse needs
  • Generation of alternative formats: tactile, audio-tactile, sub-titles etc.

Aims (2)

• Source independence
  • Use the sources authors produce
• Tool independence
  • Do not require a particular system to author or read in
• Platform independence
  • Work with all operating systems, browsers, AT

Sourcing Semantics

• STEM subjects are living languages
• New concepts created everyday
• Potentially infinite
• Trying to capture this in an declarative standard is impossible

Sources of Semantics

• Authors?
  • Never rely on authors doing their part
  • Even if they are willing they often get it wrong
• Remedial Officers?
  • Often lack the expert knowledge
  • ALT text is usually not enough!
  • Too much content to catch up
• Automatic
  • Exploit available sources
  • Use intelligent methods to automatically interpret syntax
  • Avoid manual intervention as much as possible

Maths Formulas

• How to generate accessible Maths?
• Be careful when you convert your sources...

Convert from Sources

Open source tools for converting maths documents from diverse sources

• Translate a LaTeX or Markdown document with pandoc

pandoc --standalone -t html -o example.html example.md

• Or with LaTeXML

latexml example.tex | latexmlpost --dest=example.html

• Or from Word sources

pandoc --self-contained -s example.docx -t html -o example.html

Embed Semantics

• Enrich Math using Speech Rule Engine
• Javascript library for translating XML into Speech (markup)
• Open source library available at github
• Only Math speech solution in JavaScript
• Runs in browser, command line, as node module.
• Initially implemented in the context of ChromeVox
• Speech solution for: ChromeVox, MathJax, EquatIO, MathLive, ...

Generate Semantics

• Math markup (e.g., LaTeX, MathML) are not very expressive
• SRE rewrites internally into semantically improved term structure

    ax^2+bx+c=0

<math>
  <mi>a</mi>
  <msup>
    <mi>x</mi>
    <mn>2</mn>
  </msup>
  <mo>+</mo>
  <mi>b</mi>
  <mi>x</mi>
  <mo>+</mo>
  <mi>c</mi>
  <mo>=</mo>
  <mn>0</mn>
</math>

Render Accessibly

• For example use MathJax
• A JavaScript library for rendering Mathematics in all browsers
• Input: LaTeX, AsciiMath, and MathML
• Output: HTML/CSS, SVG
• De facto rendering solution of (nearly) all Mathematics on the web
• Uses SRE for accessibility

Example

$$ x = \frac{-b \pm \sqrt{b^2-4ac}}{2a} $$

• Source in LaTeX
• Rendered with MathJax
• Embedded semantics generated with SRE
• Accessibility "built-in"

Multi-modality

• Speech or Braille output (e.g., Nemeth)
• Linear nemeth output $$ \vec{\nabla} \times \vec{F} = \left( \frac{\partial F_z}{\partial y} - \frac{\partial F_y}{\partial z} \right) \mathbf{i} + \left( \frac{\partial F_x}{\partial z} - \frac{\partial F_z}{\partial x} \right) \mathbf{j} + \left( \frac{\partial F_y}{\partial x} - \frac{\partial F_x}{\partial y} \right) \mathbf{k} $$
• Nemeth is ephemeral and times out on a Braille display
• New standards like aria-braille-label might help

LaTeX to Nemeth Project

• Project by the NFB, PreTeXt and AIM:
  • Rob Beezer (PreTeXt / Pudget Sound)
  • David Farmer (PreTeXt / AIM)
  • Alexei Kolesnikov (Towson)
  • Al Maneki (NFB)
  • Martha Siegel (Towson)
• Generating tactile Maths in Braille from PreTeXt Input.
• PreTeXt is a XML markup with LaTeX maths
• Mainly used for open source textbooks

LaTeX to Nemeth

• PreTeXt books are translated automatically into Braille with
  • Liblouis for the text
  • MathJax for formulas using SRE's Nemeth translation
• Multistep translation pipeline server-side using MathJax3
• Current pipeline is mainly a prototype
• Tactile graphics still primarily done manually

LaTeX to Nemeth (Current State)

• Two books partially translated (Abstract Algebra, Calculus)
• Bugs discovered in SRE (few fixed)
• Aim workshop in August
• Report for BANA on missing Nemeth constructs

LaTeX to Nemeth (Potential)

• Improve pipeline and Nemeth translation
• Working on 2D layout structures and diagrams
• Automate translation of graphics to tactiles
• Reasonably support transcribers
• Extend from PreTeXt to any LaTeX

DON'T Rely on Single Standards

• Try to express this in MathML: $$ \begin{prooftree} \AxiomC{} \RightLabel{$Hyp^{1}$} \UnaryInfC{$P$} \AXC{$P\to Q$} \RL{$\to_E$} \BIC{$Q^2$} \AXC{$Q\to R$} \RL{$\to_E$} \BIC{$R$} \AXC{$Q$} \RL{Rit$^2$} \UIC{$Q$} \RL{$\wedge_I$} \BIC{$Q\wedge R$} \RL{$\to_I^1$} \UIC{$P\to Q\wedge R$} \end{prooftree} $$

... or Single Sources

• Use PDF as a source
• Existing work: MaxTract (over 10 years old), Infty
• Ongoing project with the AssisTech Group IIT Delhi
• Resurrects an old project of mine: MaxTract (the good parts)
• Translate inaccessible PDF into accessible ePub
• We use pdf2htmlEX
• Builds a client side accessible rendering for PDF documents
• Runs server-side with puppeteer
• Can handle Maths, Tables, some diagrams

What about Diagrams?

• Semantic representations?
• SVG is an ideal container format:
  • Highly structured
  • Embedding of hierarchical content
  • Foreign objects embedded via namespaces
  • Muliple views on content
• Support of diverse audiences
• Multi-modal output
• Sources can be images, unique identifiers, models, etc.

Generate with Knowledge

• Easiest way to embed knowledge is, if I know what I am talking about
• Data visualisations directly generated from statistical models
• Authors do not even realise that it is accessible out of the box
• Example: R statistics package
  • BrailleR package by Jonathan Godfrey
  • I added web accessiblitiy
• Other examples: HighCharts
• Other domains: Org charts from Office software using stylesheet transformations

Example: Accessible Statistics

Get Semantics from Scratch

• Image analysis and recognition
• Alternatively: Unique Identifiers
• Generation of annotated SVG
• Semantic enrichment
  • Recursively identify major components
  • Use web services for naming
• Generate hierarchical navigation structure

Chemistry Example

Other Examples

• More examples here
• This also works as a tactile or audio-tactile graphic
• Works with ViewPlus's products
• Also for Physics

Test Live

Live web service

• Pilot project with a number of Institutions
• Project with LibreText

Combined in STEM Documents

• All this technology plays well together in single web documents
• Example: Take these slides
• Or my Report for the DIAGRAM Center.

Conclusions

• Basic mechanisms for accessible STEM on the web are available
• We just have to use them right
• Let's not be too hung-up on standards
• Always avoid limiting the creativity of authors
• We need more emphasis on
  • automatic interpretation,
  • machine learning and understanding
  • implementation of flexible workflows

Next steps

• Covid-19:
  • Organisations are forced to move content online
  • If done badly, it will be a lost opportunity!
  • Preserving the knowledge is most important!
• Education is important
• AMS training workshops: How to move Maths online
  • (tentative 31 October)