publications | Eugene Jang

2024

When faulty AI falls in the wrong hands: The risks of erroneous AI-driven healthcare decisions (In Press)

Eugene Jang

International Journal of Communication, 2024

2023

Contact-Tracing Apps as Boundary Objects of Pandemic Governance: The State-by-State Approach to Contain the Spread of COVID-19 in the United States

Eugene Jang, Jeeyun Baik, and Katrin Fischer

International Journal of Communication, 2023

Abs HTML

Digital contact tracing has been claimed as imperative to controlling the spread of COVID-19. However, the state-by-state approach in the United States led to divergences in contact tracing. This study analyzed contact-tracing apps as “boundary objects” through which each state worked toward the governance of the pandemic without having a formal consensus. Through media coverage and walkthrough analyses of three digital contact-tracing apps in Alabama, California, and New York, we closely investigated both convergences and divergences of the apps. In the process, we located the implications of Google/Apple’s Bluetooth-based exposure notification system for digital contact tracing within and beyond state boundaries. Our findings suggest that the development of apps shared the notion of an ideal contact-tracing method—exposure notification—while each state was also situated in their local experiences of the pandemic as reflected in distinct app features. We further discuss the implications of techno-solutionist standardization of such digital contact-tracing apps.
WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models

Virginia Felkner, Ho-Chun Herbert Chang, Eugene Jang, and Jonathan May

In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2023

Abs HTML

We present WinoQueer: a benchmark specifically designed to measure whether large language models (LLMs) encode biases that are harmful to the LGBTQ+ community. The benchmark is community-sourced, via application of a novel method that generates a bias benchmark from a community survey. We apply our benchmark to several popular LLMs and find that off-the-shelf models generally do exhibit considerable anti-queer bias. Finally, we show that LLM bias against a marginalized community can be somewhat mitigated by finetuning on data written about or by members of that community, and that social media text written by community members is more effective than news text written about the community by non-members. Our method for community-in-the-loop benchmark development provides a blueprint for future researchers to develop community-driven, harms-grounded LLM benchmarks for other marginalized communities.
Digital Epidemiological Approaches in HIV Research: A Scoping Methodological Review

Lindsay Young, Yuanfeixue Nan, Eugene Jang, and Robin Stevens

Current HIV/AIDS Reports, Jul 2023

Abs HTML

The purpose of this scoping review was to summarize literature regarding the use of user-generated digital data collected for non-epidemiological purposes in human immunodeficiency virus (HIV) research. Thirty-nine papers were included in the final review. Four types of digital data were used: social media data, web search queries, mobile phone data, and data from global positioning system (GPS) devices. With these data, four HIV epidemiological objectives were pursued, including disease surveillance, behavioral surveillance, assessment of public attention to HIV, and characterization of risk contexts. Approximately one-third used machine learning for classification, prediction, or topic modeling. Less than a quarter discussed the ethics of using user-generated data for epidemiological purposes. User-generated digital data can be used to monitor, predict, and contextualize HIV risk and can help disrupt trajectories of risk closer to onset. However, more attention needs to be paid to digital ethics and the direction of the field in a post-Application Programming Interface (API) world.
Who says what in which networks: What influences social media users’ emotional reactions to the COVID-19 vaccine infodemic?

Aimei Yang, Shin Jieun, Hye Min Kim, Alvin Zhou, Wenlin Liu, Ke Huang-Isherwood, Eugene Jang, Jingyi Sun, and 3 more authors

Social Science Computer Review, Jul 2023

Abs HTML

This study aims to identify effective predictors that influence publics’ emotional reactions to COVID-19 vaccine misinformation as well as corrective messages. We collected a large sample of COVID-19 vaccine related misinformation and corrective messages on Facebook as well as the users’ emotional reactions (i.e., emojis) to these messages. Focusing on three clusters of features such as messages’ linguistic features, source characteristics, and messages’ network positions, we examined whether users’ reactions to misinformation and corrective information would differ. We used random forest models to identify the most salient predictors among over 70 predictors for both types of messages. Our analysis found that for misinformation, political ideology of the message source was the most salient feature that predicted anxious and enthusiastic reactions, followed by message features that highlight personal concerns and messages’ network positions. For corrective messages, while the sources’ ideology was still key to raising anxiety, the most important feature for triggering enthusiasm was the messages’ network positions and message quality.
Measuring Online Emotional Reactions to Events

Siyi Guo, Zihao He, Ashwin Rao, Eugene Jang, Yuanfeixue Nan, Fred Morstatter, Jeffrey Brantingham, and Kristina Lerman

In Proceedings of the International Conference on Advances in Social Networks Analysis and Mining, Jul 2023

Abs HTML

The rich and dynamic information environment of social media provides researchers, policy makers, and entrepreneurs with opportunities to learn about social phenomena in a timely manner. However, using this data to understand social behavior is difficult due heterogeneity of topics and events discussed in the highly dynamic online information environment. To address these challenges, we present a method for systematically detecting and measuring emotional reactions to offline events using change point detection on the time series of collective affect, and further explaining these reactions using a transformer-based topic model. We demonstrate the utility of the method on a corpus of tweets from a large US metropolitan area between January and August, 2020, covering a period of great social change. We demonstrate that our method is able to disaggregate topics to measure population’s emotional and moral reactions. This capability allows for better monitoring of population’s reactions during crises using online data.

2022

Where horizontal and vertical surveillances meet: Sense-making of US COVID-19 contact-tracing apps during a health crisis

Jeeyun Baik, and Eugene Jang

Mobile Media & Communication, Jul 2022

Abs HTML

Analyzing user reviews of seven US digital contact-tracing apps for COVID-19, this article unpacks how the new form of surveillance technology is understood and experienced by individuals during a global health crisis. The findings suggest that the app users felt empowered via self-tracking capacity and expressed community-level care and concerns, including those regarding the marginalized. At the same time, the users were raising doubts over technical effectiveness, navigating varying levels of voluntary choice available, and negotiating privacy concerns depending on the (dis)trust they held of institutional entities behind the governance of the apps. We argue that it is critical to investigate how surveillance technologies are situated across horizontal and vertical relationships in people’s everyday lives to fully understand the individual and societal acceptance and/or refusal of the very systems during crises.
Towards WinoQueer: Developing a Benchmark for Anti-Queer Bias in Large Language Models

Virginia K Felkner, Ho-Chun Herbert Chang, Eugene Jang, and Jonathan May

arXiv preprint arXiv:2206.11484, Jul 2022

Abs HTML

This paper presents exploratory work on whether and to what extent biases against queer and trans people are encoded in large language models (LLMs) such as BERT. We also propose a method for reducing these biases in downstream tasks: finetuning the models on data written by and/or about queer people. To measure anti-queer bias, we introduce a new benchmark dataset, WinoQueer, modeled after other bias-detection benchmarks but addressing homophobic and transphobic biases. We found that BERT shows significant homophobic bias, but this bias can be mostly mitigated by finetuning BERT on a natural language corpus written by members of the LGBTQ+ community.

2019

Beautiful and Masculine: Male Make-Up YouTubers and Heteronormativity in South Korea

Eugene Jang, Sojeong Park, Jee Won Lee, and Seok-Kyeong Hong

The journal of popular culture, Jul 2019

Abs HTML

The article focuses on Korean male make-up instructors on the social media platform YouTube.com and how they represent heteronormativity in South Korea. The authors discuss the concept of masculinity in East Asian mass media, including Korean boy bands known as K-pop groups and the influence of Japanese manga, examine these challenges to traditional masculinity, and explore notions of male beauty.