Bio and Research Interests
I grew up in the beautiful Pacific Northwest, and due to my passion for the outdoors, I love it here. Feel free to see a gallery of some of my photos. I am passionate about outdoors activites such as running, paddleboarding, snowskiing, hiking, and mountain biking. I also like environmental protection, whole plant foods, following lifestyle medicine guidelines, science, and living a sustainable lifestyle. Happily, Adobe does a pretty good job in terms of sustainability.
My career has focused on computer science research with numerous Adobe and academic collaborators, and particularly on algorithms used by photographers. I have worked within computer graphics and computer vision. Recently I have worked on image tasks like segmentation for identifying distracting elements in photographs, inpainting, generative modeling, multimodal large language models, super-resolution, and efficiency and distillation topics related to those tasks. For the years 2013-2017, I was an assistant professor of computer science at the University of Virginia.
Please see my C.V. or my publications below for more information.
My career has focused on computer science research with numerous Adobe and academic collaborators, and particularly on algorithms used by photographers. I have worked within computer graphics and computer vision. Recently I have worked on image tasks like segmentation for identifying distracting elements in photographs, inpainting, generative modeling, multimodal large language models, super-resolution, and efficiency and distillation topics related to those tasks. For the years 2013-2017, I was an assistant professor of computer science at the University of Virginia.
Please see my C.V. or my publications below for more information.
Internships
I will be recruiting computer science Ph.D. students for Adobe research internships, which typically result in peer-reviewed publications. If you are interested, please apply. I suggest you send myself and any other researchers with overlapping research interest an email. It helps if you can identify shared research interests. For summer internships, it also helps if you do this in November.
Service
I have served on these technical papers committees: CGDIP (2017), Eurographics short papers (2013), Eurographics Symposium on Rendering (2013, 2014, 2017), ICCP (2013), Pacific Graphics (2017), SIGGRAPH (2015, 2016, 2019, 2020), SIGGRAPH Asia (2018, 2021, 2024).
Classes
If you are interested, you can see a list of classes that I taught when I was a professor.
Impact
The PatchMatch technology that I researched for my Ph.D. was incorporated into Adobe Photoshop CS5 as content-aware fill, which has now largely been replaced by generative AI, but is still used in Adobe Lr/Ps for certain removal cases. This received 8 million views on YouTube [1] [2], and was featured in press such as Popular Science and PC Magazine. In later Photoshop versions, Content-aware Patch, Move, and Color Adaptation were built on this technology [3]. More recently, I researched various computer vision technologies for analysis and generative AI including GANs and diffusion models, which have been integrated into Photoshop's new remove tool. I also recently worked on automatic wire removal in Ps, fast image generation using few-step diffusion model distillation, and super-resolution such as enhance detail in Ps for generative fill.
Towards the beginning of my career, I also did some early research on the Halide project, which has been used at Adobe and Google.
Research Publications (Google
Scholar)
I have indicated a few of my favorite papers with asterisks (*).
Diffusion2GAN: Distilling Diffusion Models into Conditional GANs
ECCV 2024
Brush2Prompt: Contextual Prompt Generator for Object Inpainting
CVPR 2024
Perceptual Artifacts Localization for Image Synthesis Tasks
ICCV 2023
Automatic High Resolution Wire Segmentation and Removal
CVPR 2023
SimpSON: Simplifying Photo Cleanup With Single-Click Distracting Object Segmentation Network
CVPR 2023
Inpainting at Modern Camera Resolution by Guided PatchMatch with Auto-Curation
ECCV 2022
Image Inpainting with Cascaded Modulation GAN and Object-Aware Training
ECCV 2022
Perceptual Artifacts Localization for Inpainting
ECCV 2022, oral presentation
Deep 360° Optical Flow Estimation by Multi-Projection Fusion
ECCV 2022
Aδ: Autodiff for Discontinuous Programs - Applied to Shaders *
SIGGRAPH 2022
Learning from Shader Program Traces *
Eurographics 2022. Best full paper award.
Searching for Fast Demosaicking Algorithms
ACM Transactions on Graphics (presented at SIGGRAPH 2022)
Modulated Periodic Activations for
Generalizable Local Functional Representations
ICCV 2021
TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations
CVPR 2021
Coherent Video Generation for Multiple Hand-held Cameras with Dynamic Foreground
Computational Visual Media (CVM) 2020
Unselfie: Translating Selfies to Neutral-pose Portraits in the Wild
ECCV 2020
Image Morphing With Perceptual Constraints and STN Alignment
Eurographics 2020
Learning to Generate Textures on 3D Meshes
CVPR Workshops: 3D WiDGET 2019. Best paper award.
On the Continuity of Rotation Representations in Neural Networks *
CVPR 2019
Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture
CVPR 2019
Approximate Program Smoothing Using Mean-Variance Statistics, with Application to Procedural Shader Bandlimiting *
Eurographics 2018
Where and Who? Automatic Semantic-Aware Person Composition
IEEE Winter Conference on Applications of Computer Vision (WACV) 2018
Automatic Image Defect Diagnosis
IEEE Winter Conference on Applications of Computer Vision (WACV) 2018
Halide: A Language and Compiler for Optimizing Parallelism, Locality, and Recomputation in Image Processing Pipelines
Communications of the ACM 2018: Research Highlights
Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
arXiv preprint, 2017
VizGen: Accelerating Visual Computing Prototypes in Dynamic Languages
ACM SIGGRAPH Asia 2016
Image Perforation: Automatically Accelerating Image Pipelines by Intelligently Skipping Samples
ACM Transactions on Graphics 2016 (to appear at ACM SIGGRAPH 2016)
Towards Automatic Band-Limited Procedural Shaders
Pacific Graphics 2015
PatchTable: Efficient Patch Queries for Large Datasets and Applications
ACM SIGGRAPH 2015
Synthesis of Complex Image Appearance from Limited Exemplars
ACM Transactions on Graphics 2015 (presented at ACM SIGGRAPH)
RealPigment: Paint Compositing by Example
NPAR 2014: Symposium on Non-Photorealistic Animation and Rendering
Stylized Keyframe Animation of Fluid Simulations
NPAR 2014: Symposium on Non-Photorealistic Animation and Rendering
Style Transfer for Headshot Portraits
ACM SIGGRAPH 2014
DecoBrush: Drawing Structured Decorative Patterns by Example
ACM SIGGRAPH 2014
Camouflaging an Object from Many Viewpoints
CVPR 2014 (oral presentation)
Patch-based High Dynamic Range Video
ACM SIGGRAPH Asia 2013
RealBrush: Painting with Examples of
Physical Media
ACM SIGGRAPH 2013
Halide: A Language and Compiler for Optimizing Parallelism, Locality and Recomputation in Image Processing Pipelines *
ACM PLDI 2013
Image Melding: Combining Inconsistent Images using Patch-based Synthesis
ACM SIGGRAPH 2012
The PatchMatch Randomized Matching Algorithm for Image Manipulation
Communications of the ACM: Research Highlights, 2011
PatchMatch: A Fast Randomized Matching Algorithm with Application to Image and Video
Ph.D. Dissertation, Princeton University, 2011
PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing *
ACM SIGGRAPH 2009
Video Puppetry: A Performative Interface for Cutout Animation
. + Equal contribution
ACM SIGGRAPH Asia 2008
Education Publications
Enhancement of Student Learning in Experimental Design Using a Virtual Laboratory
IEEE Transactions on Education, 2008
Experiential Learning of Design of Experiments Using a Virtual CVD Reactor
American Society for Engineering Education Conference, 2006. Award for best paper in Chemical Engineering.
The Virtual CVD Learning Platform
Frontiers in Education Conference, 2006.
ThermoSolver: An Integrated Educational Thermodynamics Software Program
Undergraduate honors thesis, 2006. Oregon State University Library
Other Resources
My personal website includes other resources such as:
- Source code, including PatchMatch code and Python bindings for the Halide language.
- Artwork
- Poetry
- Writings