The Team

nirupam Nirupam Roy

Director, Asst. Prof.

nakul Nakul Garg

Ph.D. Student

yang Yang Bai

Ph.D. Student

irtaza Irtaza Shahid

Ph.D. Student

harsh Harsh Takawale

Ph.D. Student

The Team

aritrik Aritrik Ghosh

Ph.D. Student

ayushi Ayushi Mishra

Ph.D. Student

sumbul Sumbul Zehra

MS Student

varun Varun Ramani

MS Student

robert Robert Estan

UG Student

Current News

Nakul is among the CPS Rising Stars 2024.
Yang receives the Outstanding Graduate Assistant Award for the academic year 2023-24 at UMD.
Project Sirius wins the MobiSys 2023 Best Poster Award.
Project SPiDR is selected for the SIGMOBILE research highlights.
Nirupam receives NSF CAREER award for Acoustic Ambient Computing.
Nirupam receives the Meta Research Award for "Physical Context-Aware Voice Assistant for Smart Homes."
Nakul receives 2023 Ann Wylie Dissertation Fellowship.
Project SPiDR wins best paper award at MobiSys'22.
LidarPhone wins best poster runner-up award at SenSys'20.
Nirupam co-chairs EarComp 2023 Techinical Program Committee.
Nirupam co-organizes Intelligent Acoustics workshop IASA 2023 at CPS-IoT week.
Nakul accepts MSR Redmond, Irtaza accepts Nokia Bell labs, and Yang accepts Apple for Summer internships 2023.
Nakul is selected for the Clark School Future Faculty Program.
Yang and Nirupam organize a Diversity & Inclusion panel at HotMobile 2023.
Project Owlet wins the best demo award at MobiSys'21.
Mahir gets the departmental honors in CS, UMD.
Yang receives N2Women fellowship 2021.

Projects

Sirius: RF Self-localization

Sirius is a self-localization system using a single receiver for low-power IoT nodes. It explores the use of gain-pattern reconfigurable antennas with passive envelope detector-based radios to perform AoA estimation without requiring any kind of synchronization.

Best poster award

Side channel defense on embedded devices

ThermWare is a non-intrusive screening method to detect anomalous operations on embedded devices at run-time. It analyses subtle heat patterns of electronic circuits during code execution to detect any malicious activities.

Making microphones hear inaudible sounds

Backdoor shows that these high frequency sounds can be designed to become recordable by unmodified microphones, while remaining inaudible to humans.Applications include short-range data communication, jamming spy microphones in the environment, live watermarking of music in a concert, and even acoustic denial-of-service (DoS) attacks and inaudible voice command attacks to voice assistants.

Best paper award

Brain injury detection in sports

Heading the ball is a fundamental skill in soccer. This project develops a method to assess the impact of head-ball contacts using the Adidas micoach soccer ball and an on-body inertial sensor.

Student research competition winner

Preventing DeepFake attacks on Live Speech

Malicious editing of audiovisual content has emerged as a popular tool for targeted defamation, spreading disinformation, and triggering political unrest. TalkLock is a live speech integrity verification system that can enable live speakers to protect their speeches from malicious alterations even when the speech is recorded by any member of the audience.

Communication through vibrations

Data communication has been studied over a wide range of modalities, including radio frequency (RF), acoustic, visible light, etc. This project envisions vibration as a new mode of secure short-range communication through physical touch. We engineer a completely functional prototype, which entails a full OFDM stack, coping with ADC saturation, synchronization, error coding, and interleaving.

Reconfigurable Robotic Network Infrastructure

Mobile computing has traditionally implied mobile clients connected to a static infrastructure. This project breaks away from this point of view and envisions the possibility of injecting mobility into infrastructure.

Privacy attacks using mobile sensor fingerprints

This projects shows that the hardware imperfections leads to identifiable unique features in sensor data called fingerprints. Attackers can leverage such fingerprints to track smartphones.

Best poster runner-up award

SPiDR: Ultra-low-power acoustic imaging

SPiDR is an ultra-low-power acoustic perception system that produces a cross-sectional map of the field-of-view using only one speaker/microphone pair. It leverages sound’s interaction with small structures to project spatially coded signals for imaging.

Best paper award

Low-power situational awareness for small drones

This project is our first step toward a long-term research commitment focused around developing low-latency sensing-inferencing mechanism to defend autonomous systems against high-speed dynamic obstacles and projectile objects. We particularly target SWAP-C constrained miniature robots and drones.

Spying on speech through a robot vacuum cleaner's LIDAR

This project presents LidarPhone, a novel acoustic side-channel attack through the LIDAR sensors equipped in popular commodity robot vacuum cleaners. The core idea is to repurpose the lidar to a laser-based microphone that can sense sounds from subtle vibrations induced on nearby objects. LidarPhone carefully processes and extracts traces of sound signals from inherently noisy laser reflections to capture privacy sensitive information.

Best poster runner-up award

Recovering speech from vibrations

VibraPhone demonstrates the feasibility of using the vibration motor in mobile devices as a sound sensor, almost like a microphone.

Owlet: Low-power and miniaturized spatial sensing

Owlet is a low-power and miniaturized system for measuring spatial information from sound. Owlet uses acoustic microstructures to embed direction-specific signatures on the recorded sound and develops a transfer learning-based approach signal direction estimation and localization in real-time.

Best demo award

Disaster resilience through wireless infrastructure

Urban greenspaces are untapped assets in infrastructure improvement and disaster mitigation. Project BRIDGE augments greenspace with broadband infrastructure to mitigate digital divide and improve community resilience.

Noise cancellation using wireless-acoustic co-design

Active Noise Cancellation (ANC) is a classical area where noise in the environment is canceled by producing anti-noise signals near the human ears (e.g., in Bose’s noise cancellation headphones). This paper brings IoT to active noise cancellation by combining wireless communication with acoustics.

Indoor localization without infrastructure

WalkCompass is a system that exploits smartphone sensors to estimate the direction in which a user is walking. This work provides a crcuial building block for infrastructre less indoor localization. WalkCompass develops a stable technique to estimate the user’s walking direction within a few steps.

Outstanding MS thesis award

People

Awards & Honors

CPS Rising Stars 2024
Nakul, the final year Ph.D. student at iCoSMoS, has been selected as one of the CPS Rising Stars, 2024

Best poster award, MobiSys 2023
Sirius, a solution for ultra-low-power self-localization on resource constrained devices, wins best poster award at MobiSys 2023.

Best paper award, MobiSys 2022
SPiDR, an ultra-low-power spatial sensing system for miniature mobile robots, wins best paper award at MobiSys 2022.

Best demo award, MobiSys 2021
Owlet, the low-power and miniaturized spatial sensing system, wins best demo award at MobiSys 2021.

Best poster runner-up award, SenSys 2020
LidarPhone wins best poster runner-up award at SenSys 2020. LidarPhone shows how the lidar sensor can be exploited to extract traces of spoken words and uncovers an acoustic eavesdropping threat.

CSL Ph.D. Thesis Award, 2019
The thesis “Inaudible Acoustics: Techniques and Applications” wins the CSL Ph.D. thesis award 2019 at UIUC.

Best paper award at MobiSys, 2017
“BackDoor: Making Microphones Hear Inaudible Sounds” wins the best paper award and the audience-choice award at MobiSys 2017.

Qualcomm Innovation Fellowship (QInF) finalist, 2016
“The QInF program is focused on recognizing, rewarding, and mentoring innovative PhD students across a broad range of technical research areas, based on Qualcomm’s core values of innovation, execution and teamwork.”

Bahl fellowship as outstanding graduate student, 2015-2017
“The fellowship was established by Dr. Lalit R. Bahl and Kavita Kinra and will provide graduate fellowships to students in the Electrical and Computer Engineering Department, UIUC”

Valkenburg award for outstanding research, 2015
“Mac Van Valkenburg symbolizes the highest quality of engineering education in Illinois and, indeed, in the country. The award, which is given annually to one or two outstanding PhD candidates in ECE, UIUC.”

Outstanding thesis award for MS thesis, 2013
The MS thesis was selected for the outstanding thesis award 2013 at the University of South Carolina, Columbia.

Best poster/demo runner-up at HotMobile, 2013
The poster on smartphone fingerprinting (AccelPrint) won the second prize at the HotMobile 2013, the International Workshop on Mobile Computing Systems and Applications.

Publications

Papers

NSDI 2025

Large Network UWB Localization: Algorithms and Implementation
Nakul Garg (UMD), Irtaza Shahid(UMD), Ramanujan K Sheshadri (NEC Labs), Karthik Sundaresan (GATech), Nirupam Roy (UMD)

Coming soon!

UbiComp/IMWUT 2024

Scribe: Simultaneous Voice and Handwriting Interface
Yang Bai, Irtaza Shahid, Harshvardhan Takawale, Nirupam Roy

GetMobile 2024

Microstructure-Assisted Vision: Adding New Senses to Low-Power Devices
Yang Bai, Nakul Garg, Nirupam Roy
SIGMOBILE research highlights

MobiSys 2023

HotMobile 2023

ThermWare: Toward Side-channel Defense for Tiny IoT Devices
Nakul Garg, Irtaza Shahid, Erin Avllazagaj, Jennie Hill, Jun Han, Nirupam Roy

IASA 2023

Structure Assisted Spectrum Sensing for Low-power Acoustic Event Detection
Nakul Garg, Harshvardhan Takawale, Yang Bai, Irtaza Shahid, Nirupam Roy

MobiSys 2022

SPiDR: Ultra-low-power Acoustic Spatial Sensing for Micro-robot Navigation
Yang Bai (co-primary), Nakul Garg (co-primary), Nirupam Roy
Best paper award

IASA 2022

VoiceFind: Noise-Resilient Speech Recovery in Commodity Headphones
Irtaza Shahid, Yang Bai, Nakul Garg, Nirupam Roy

AVAR 2022

Personal active soundfield control (PASCAL) framework
Shoken Kaneko, Irtaza Shahid, Nirupam Roy, Nail Gumerov, Ramani Duraiswami

MobiSys 2021

Owlet: enabling spatial information in ubiquitous acoustic devices
Nakul Garg (co-primary), Yang Bai (co-primary), Nirupam Roy
Best demo award

HotMobile 2020

SenSys 2020

Spying with Your Robot Vacuum Cleaner: Eavesdropping via Lidar Sensors
Sriram Sami, Yimin Dai, Sean Rui Xiang Tan, Nirupam Roy, Jun Han
Best poster runner-up award

Ph.D. Dissertation

Inaudible Acoustics: Techniques and Applications
Nirupam Roy
CSL Ph.D. thesis award, UIUC  

SIGCOMM 2018

MUTE: Bringing IoT to Noise Cancellation
Sheng Shen, Nirupam Roy, Junfeng Guan, Haitham Hassanieh, Romit Roy Choudhury

NSDI 2018

Inaudible Voice Commands: The Long-Range Attack and Defense
Nirupam Roy, Sheng Shen, Haitham Hassanieh, Romit Roy Choudhury

MobiSys 2017

BackDoor: Making Microphones Hear Inaudible Sounds
Nirupam Roy, Haitham Hassanieh, Romit Roy Choudhury
Best paper award   Audience choice award

MobiSys 2016

Listening through a Vibration Motor
Nirupam Roy, Romit Roy Choudhury

ICNP 2016

Compressing Backoff in CSMA Networks
Mahanth K Gowda, Nirupam Roy, Romit Roy Choudhury, Srihari Nelakuditi

NSDI 2015

Ripple: Communicating through Physical Vibrations
Nirupam Roy, Mahanth K Gowda, Romit Roy Choudhury

MobiSys 2014

I am a Smartphone and I can Tell my User’s Walking Direction
Nirupam Roy, He Wang, Romit Roy Choudhury
Outstanding MS thesis award

NDSS 2014

AccelPrint: Imperfections of Accelerometers Make Smartphones Trackable
Nirupam Roy, S. Dey (Co-primary), W. Xu, R. Roy Choudhury, S. Nelakuditi

HotNets 2014

Infrastructure Mobility: A What-if Analysis
Mahanth K Gowda, Nirupam Roy, Romit Roy Choudhury

HotWireless 2014

Backing out of Linear Backoff in Wireless Networks
Mahanth K Gowda, Nirupam Roy, Romit Roy Choudhury, Srihari Nelakuditi

Poster/Demo

MobiSys 2023

Poster: Ultra-low-power Angle-of-Arrival Estimation Using a Single Antenna
Nakul Garg, Nirupam Roy
Best poster award

MobiSys 2023

Poster: Preventing Fake News through Live Speech Signature
Irtaza Shahid, Nirupam Roy

HotMobile 2023

Poster: Bringing AR/VR to Everyday Life - A Wireless Localization perspective
Nakul Garg, Irtaza Shahid, Karthik Sankar, Mallesham Dasari, Ramanujan Sheshadri, Karthikeyan Sundaresan, Nirupam Roy

HotMobile 2023

Poster: Natural Voice Interface for the Next Generation of Smart Spaces
Yang Bai, Nakul Garg, Harshvardhan Takawale, Anupam Das, Nirupam Roy

MobiSys 2022

Poster: Ultra-low-power Acoustic Imaging
Yang Bai (co-primary), Nakul Garg (co-primary), Nirupam Roy

MobiSys 2021

Demo: Microstructure-guided Spatial Sensing for Low-power IoT
Nakul Garg (co-primary), Yang Bai (co-primary), Nirupam Roy
Best demo award

HotMobile 2020

Poster: Acoustic Sensing for Detecting Projectile Attacks on Small Drones
Nakul Garg, Nirupam Roy

SenSys 2020

Poster: LidarPhone: Acoustic Eavesdropping using a Lidar Sensor
Sriram Sami, Sean Rui Xiang Tan, Yimin Dai, Nirupam Roy, Jun Han
Best poster runner-up award

MobiCom 2018

Poster: Networked Acoustics Around Human Ears
Sheng Shen, Nirupam Roy, Junfeg Guan, Haitham Hassanieh, Romit Roy Choudhury

MobiSys 2017

Demo: Riding the Non-linearities to Record Ultrasound with Smartphones
Nirupam Roy, Haitham Hassanieh, Romit Roy Choudhury

MobiCom 2016

Assessing Header Impacts in Soccer with Smartball
Theodore Stone et. al., Mentor: Nirupam Roy
Student research competition winner

NSDI 2016

Vibration: A new mode of communication
Nirupam Roy, Romit Roy Choudhury

Google 2015

Ripple: Communicating through Physical Vibration
Nirupam Roy, Romit Roy Choudhury
Google Mobile Faculty Summit 2015

MobiSys 2014

Demo: I am a Smartphone and I can Tell my User’s Walking Direction
Nirupam Roy, He Wang, Romit Roy Choudhury

HotMobile 2013

Leveraging Imperfections of Sensors for Fingerprinting Smartphones
Sanorita Dey, Nirupam Roy, Wenyuan Xu, Srihari Nelakuditi
Best poster runner-up award

Patents

Pending

Audio control system
Ramani Duraiswami, Shoken Eckhart Kaneko, Nail A. Gumerov, Nirupam Roy
Patent number: US 20220377461 A1,   Nov 24, 2022

Granted

Causing a voice enabled device to defend against inaudible signal attacks
Nirupam Roy, Romit Roy Choudhury, Haitham Al Hassanieh
Patent number: US 11264047 B2,   March 1, 2022

Granted

Vibrational devices as sound sensors
Romit Roy Choudhury, Nirupam Roy
Patent number: US 10628484 B2,   April 21, 2020

Granted

Unsupervised indoor localization and heading directions estimation
Romit Roy Choudhury, He Wang, Nirupam Roy, S. Sen, M. Youssef, A. Elgohary, M. Farid
Patent number: US 9730029 B2,   Aug 8, 2017

Granted

Communicating through physical vibration
Nirupam Roy, Romit Roy Choudhury, Mahanth K. Gowda
Patent number: US 20150281910 A1,   March 28, 2017

News & Media Coverage

AccelPrint


BackDoor


VibraPhone



LipRead



MUTE