Podcast Success Index: A New Era for Academic Publishing

Gandhinagar: Ever wish you could skip the academic jargon, while reading a research paper, and get straight to the heart of cutting-edge finding? Scholarly podcasting may just be that answer!

Traditionally, academics shared their findings primarily through peer-reviewed journals. While important, these journals often only reach a limited audience within academia. Scholarly podcasting is trying to break down those walls, offering researchers a dynamic platform to engage with a global audience of fellow academics, professionals, students, and anyone curious to learn!

New research called “Broadening Legitimacy of Scholarly Podcasting as Knowledge Dissemination: Metrics, Opportunities and Considerations” highlights the need for fresh approaches to measure the impact of scholarly podcasts—transforming how new research is shared and even how we measure the impact of research. 

The study was conducted by Lindsay Persohn from the University of South Florida and Stephanie Branson from Northern Arizona University. Both are involved in hosting and producing a podcast, Classroom Caffeine, aimed at connecting education researchers and teachers.

But a big question looms; how do we measure the success of these podcasts, especially in a system that prioritizes publications and citations?  Traditional metrics don’t quite fit this new format. Downloads offer a glimpse into a podcast’s reach, but they don’t tell the whole story. How do we gauge whether a podcast has sparked new research projects or influenced teaching methods?

Lindsay and Stephanie highlight the need for fresh approaches to measure the impact of scholarly podcasts.

• Podcast Success Index (PSI): This metric, developed by Singh et al. (2016), calculates a podcast’s success based on its longevity (in months) and the number of episodes produced per month. It provides a standardized measure of a podcast’s longevity and production frequency, calculated using the following formula:

PSI = log [(episodes/month) √ months active]

But it does not account for download numbers, listener engagement, or impact; may favor older podcasts.

• Downloads: The number of times podcast episodes are downloaded offers insights into the podcast’s reach and audience engagement. Podcast hosting platforms typically provide detailed download statistics, allowing podcasters to track trends over time. This metric can be considered analogous to article reads in traditional publishing.
• Geographic reach: Data on the locations of podcast listeners, often visualized through maps, demonstrates the podcast’s ability to reach a geographically diverse audience beyond traditional academic circles.
• Website analytics: If a podcast has an accompanying website, data on website visits, page views, and referral sources (e.g., search engines, social media) provide additional insights into audience engagement and the podcast’s online presence. Tools like Google Search Console can track how users find the website through search engines.

Listener feedback surveys and reviews: Gathering feedback directly from listeners offers valuable insights into the listener experience and the perceived value of the podcast. Survey data can reveal listener demographics, motivations for listening, and how they utilize the information presented in the podcast. Qualitative reviews on platforms like Apple Podcasts can provide anecdotal evidence of listener engagement and impact .

The research paper focuses on a single case study of an education-focused podcast called Classroom Caffeine, hosted and produced by the authors themselves.

To measure the success of Classroom Caffeine, the authors, Lindsay and Stephanie utilized the following methods:

• PSI Calculation: The PSI for Classroom Caffeine was calculated to be 1.16, significantly higher than the average PSI of anesthesiology podcasts analyzed in the study that developed the PSI. This finding suggested that the Classroom Caffeine podcast could be considered successful based on this metric.
• Podcast Analytics: Data from the podcast hosting platform (Buzzsprout) revealed that Classroom Caffeine had over 23,000 downloads, reaching listeners in 90 countries or territories across six continents. This data showcased the podcast’s impressive geographic reach and potential for global impact.
• Website Analytics: Data from the podcast’s website (hosted on Squarespace) showed 5,890 visits from 4,761 unique visitors in a single calendar year, with the majority of visitors coming from the United States. This data, along with information from Google Search Console, demonstrated the podcast’s online presence and the different ways listeners discovered the podcast.
• Public Measures of Success: The website ListenNotes.com, which aggregates data on millions of podcasts, assigned Classroom Caffeine a listen score of 28, placing it in the top 10% of podcasts tracked on the platform.
• Listener Survey Data: The authors conducted a listener feedback survey, which revealed that respondents found the podcast valuable for professional development and felt it provided insights into educational research. All respondents reported considering the podcast a part of their professional learning.

The case study data suggested that scholarly podcasting can effectively reach a wide audience and contribute to knowledge dissemination. However, the lack of a universally understood and accepted measure for podcast success poses challenges for wider adoption and recognition within academia.

The PSI, when combined with data on downloads, geographic reach, and listener feedback, can be a useful tool for communicating the value of a scholarly podcast to institutions.

The research paper also highlights several considerations and future directions for measuring and promoting the value of scholarly podcasting

• Challenges of Tracking Public Engagement: While traditional citation metrics can track how research is used within academia, measuring public engagement and the impact of scholarly podcasts on practice and policy remains a challenge.
• Potential of DOIs for Tracking: Using DOIs (Digital Object Identifiers) for individual podcast episodes can help track citations and connect these creative works with their producers, offering a potential solution for tracking podcast impact in more traditional academic terms.
• Leveraging Existing Structures: The researchers suggest universities could play a key role in promoting scholarly podcasting by providing resources, training, and support for faculty interested in this medium
• Building Supportive Networks: The study highlights the value of podcast networks, such as the Amplify Podcast Network, in connecting scholars, fostering collaboration, advocating for the medium, and establishing standards for quality and rigor
• Fostering Interactivity and Accessibility: The sources discuss the potential of interactive elements, like quizzes and polls, to enhance listener engagement and accessibility features, like transcripts and audio descriptions, to ensure inclusivity.
• Embracing New Technologies: Suggestions to explore the possibilities of incorporating video, animation, virtual reality, and AI into podcasts to create more immersive learning experiences.
• Addressing Sustainability: The study also acknowledged the need to address the financial and time commitment required to create high-quality podcasts. Solutions may include crowdfunding, partnerships with institutions, sponsorships, and a focus on time management and community support.
• Peer Review for Scholarly Podcasts: The study also pointed towards the emergence of peer-reviewed podcast networks, such as the Amplify Podcast Network, which offers a potential model for ensuring quality control and enhancing the legitimacy of scholarly podcasting within academia.
• Evolving Institutional Standards: For scholarly podcasting to be fully embraced as a legitimate form of scholarship, institutional standards and metrics need to evolve to recognize and value this form of public knowledge sharing.