Thoughts on Auto-captioning

I am Deaf/Hard of Hearing so I am very invested in accessibility. I write this from my personal experience as a person who grew up Hard of Hearing (HOH) with spoken English as my first language. Over time, my hearing loss has progressed and I am profoundly Deaf, but still communicate primarily in spoken English with hearing people using hearing aids, lipreading, written notes, and captions.

All Deaf people do not have the same opinion on accessibility issues. Although my thoughts below are centered around captions, I want to note the importance of American Sign Language (ASL) to the Deaf community. ASL is a rich visual language that is conceptually and grammatically distinct from English. It is not a lesser or gestural reproduction of English. Captions do not provide access for many Deaf people whose primary language is ASL.

It is also important to distinguish the different types of captions. There are closed captions, which you typically see on television programs, which include descriptions of the sounds, such as “door slams” or “birds chirping.” There is CART (Communication Access Realtime Translation) in which a trained professional is transcribing what is being said instantaneously. Last, there is auto-transcription, which is voice to text computer programs that “hear” the sound and generate the corresponding text into captions. Words that sound alike may be incorrectly transcribed. Auto-transcription is what you would most typically see on services such as Zoom.

The issue of automatic captions like you see on Zoom can raise mixed feelings among Deaf and HOH people. Some people understandably feel that having auto transcription is not full access because there will inevitably be mistakes in the captioning – sometimes ones that are unfortunate, offensive, or that result in incorrect information. That makes me think of the Zoom Inclusion Ministry point that if we are accepting inferior service for some, it's not a very loving message to them. I appreciate and respect that perspective. CART live captioners will almost always provide superior captions than auto-transcription programs. Additionally, because some Deaf people do not use English as their primary language, ASL interpreters provide much better access for them than captions. Claiming a church service or event is accessible to Deaf and Hard of Hearing people because there is auto-transcription is an overstatement at best.

However, all organizations may not have the resources to provide CART or ASL interpreters, and especially when there is a free alternative of auto transcription. As a person who grew up without access to captions at all, auto-transcription is much better than nothing. It is an imperfect, but life-changing technology for me.

In my opinion as a Deaf person whose first language is English, auto-transcription for a church service is generally very good at capturing a typical minister's sermon. Sermons are typically given by someone who is skilled at speaking clearly. They are wearing microphones that allow for better transcription. Captions are much less effective for:

1. Music. Captions will often be nonsense.
   Suggestion: For music produced live in the physical sanctuary, screen share song lyrics on the screen and turn off captions during songs. For music videos, choose videos with lyrics already on the screen. As a side note, I immensely appreciate that the music director at my church provides context for the songs, such as who wrote it, when, and why it was chosen for the service. This is an inclusive practice, and enhances my experience at the service. Music is more than just sounds.
2. Comments from children during the Time for All Ages part of the service
   As Ben Ogilvie from Starr King UU Church notes, this is problematic anyway, as adults often laugh inappropriately at children’s remarks. RE folk are moving away from “children as entertainment” during worship.
3. Anything said in the physical sanctuary without a microphone
   This applies to services where congregants may offer words from their seats. If the speech doesn’t come through on Zoom, the auto-captioner program doesn’t have anything to work with. This impacts both hearing and Deaf HOH participants.
4. Videos played during the service from the sanctuary
   Suggestion: Play videos through a zoom screen share with captions. Project the screen share for Roomies.
5. Various voices
   Some voices are not accurately transcribed and this is not the speaker’s fault! It’s a limitation of today’s auto transcription technology. The ideal solution does not exist yet. We want all speakers and Deaf HOH participants to feel equally valued and supported.

So, it's not perfect. I know that some hearing people have complained when captions are turned on in the sanctuary screen because they feel the captions or the captioning errors are “distracting.” I find that perspective frustrating because having no captions really does limit my access to information and they have the option to look away– I don't have the option to hear better. I would not attend a virtual service without zoom, and I do watch the captions if I am in person at the church. I also would personally prefer it if people did not tell me that others complained about the captions. There’s something disheartening about being told something that is important to you is annoying to others who are privileged in this situation.

Other virtual events (such as small groups or committees) are generally sufficient with auto captions if there is one person speaking at a time. It is not always clear to me who is speaking and that can be harder – that's important for a lip reader like me. It can also be important to know who is saying what. Honestly, most of the issues I have had are in small groups rather than services. In services, it is typically very clear to me who is speaking because they are at the pulpit.

In group meetings, I have asked attendees to say my name first when asking me a question. For example, "Sarah, do you know X?" is better than "Do you know X?" or even "Do you know X, Sarah?" That cues me to the question and a need to respond. This can even be helpful if they are asking another attendee a question because it alerts me where to look next and the context. Not everyone in the Deaf HOH community agrees with this, but I do find it helpful.

Ben Ogilvie asked me if having my phone or tablet with a zoom connection (with video only, to avoid audio feedback) would work so I’d have captions right in front of me, just for me. I have used captions on a personal device in some locations. My concern is that it puts a bit of the burden on the person who needs the caption to set up, etc. Also, it can feel like you are outside of the group if you are holding your tablet while others are doing some meditation or something like that. I also have felt self conscious that it looks like I am on my phone during the service. It requires some navigating between device and church throughout. Not perfect, but doable. My preference is captions in front of the church so I can look relatively seamlessly between the speaker and the captions.

Tips for Hearing Folk

Tips for the Person Speaking

• Enunciate clearly and don’t rush. That will help the auto transcription a lot.
• When addressing someone in a Zoom meeting, it can be helpful to say their name first.
• Remember captioning has a slight delay between the words being spoken and captions being shown on the screen.
• Avoid remarking on captioning errors unless there’s an important error in the captions. Deaf people see these errors all the time, and comments about them get old. It's like the 100th time someone asks a tall person "How's the weather up there?" Captions are tools to support access, and not sources of entertainment for hearing people. Frequent commenting on the captions could be seen as similar as repeatedly remarking on a noise a wheelchair makes–there’s no need to unnecessarily draw attention.
• Face your camera when speaking.
• Offer other options when using language such as suggesting people “close their eyes” during a mediation part of the service, especially if there are instructions to be followed. It’s much harder to read captioned text with your eyes closed!

Tips for the Zoom Host

• Configure Zoom to enable captioning and test it before your meeting. See " Enabling Captioning in Zoom" below for details.
• Keep an eye on the captions for accuracy – or enlist someone else to – and correct if appropriate (e.g. host sees the caption calls someone Jayden when it's Brayden. 19 vs. 90 etc.)
• Encourage use of the chat for commenting because the captioning delay can make it hard for Deaf/HOH folk to comment in a timely way.
• Remind people to look at the camera when speaking.

Enabling Captioning in Zoom

Enabling Captioning by Zoom Hosts

In the configuration settings in your zoom.us account, under “In Meetings (Advanced),” turn on Automated captions and Full transcript. This allows Zoom participants to enable captioning during the meeting. You’ll need to do this (once) before the meeting! It can’t be done after the meeting has started. (OK… It can.. but it won’t affect a meeting that has already begun.)

Test it! If you can’t turn on the captions, the participants likely can’t either. Some hosts also keep the captions on during their meeting and provide corrections (sometimes in private chat feature) if they notice that a caption is incorrect and easily correctable (e.g. caption says “90 dollars” but it’s “19 dollars)

If you expect to ever have a sign language interpreter, you can enable it in the settings:

Enabling Captioning for Participants

If the Zoom host has enabled captioning in their configuration, then you can turn it on during the meeting (assuming you have a late enough version of Zoom). How to do that varies from one platform to another. Look for a Zoom control labeled “Captions” or “Show Captions.” Click or tap that and select the language you’re using. Note that this will set the language for everyone in the meeting. Then (at least on a computer) you can choose whether to have captions appear at the bottom of the screen or “View full transcript” which opens a side panel with caption text scrolling by there.