Distortion Reduction for Off-Center Perspective Projection of Panoramas

Chi-Han Peng and Jiayao Zhang

A single Panorama can be drawn perspectively without distortions in arbitrary viewing directions and field-of-views when the camera position is at the origin. This is a key advantage in VR and virtual tour applications because it enables the user to freely”look around” in a virtual world with just a single panorama, albeit at a fixed position. However, when the camera moves away from the center, barrel distortions appear and realism breaks. We propose modifications to the equirectangular-to-perspective(E2P) projection that significantly reduce distortions when the camera position is away from the origin. This enables users to not only “look around” but also “walk around” virtually in a single panorama with more convincing renderings. We compare with other techniques that aim to augment panoramas with 3D information, including: 1) panoramas with depth information and 2) panoramas augmented with room layouts, and show that our approach provides more visually convincing results.

Reconstructing 3D Indoor Layout from Multiple Panoramic Images (結合深度學習與圖形最佳化方法之多視角室內全景影像三維格局重建)

Sio-Keong Si, Jheng-Wei Su, Chi-Han Peng, Kuo-Wei Chen, Felix Chang, Chih-Yuan Yao, and Hung-Kuo Chu
Computer Graphics Workshop (CGW), 2021. *Best Paper Award
Paper (Author’s version)

We propose a novel framework to estimate room layouts from multiple panoramas taken inside the same room with registration. Our solution consists of the following major components. First, we propose a boxification line prediction network that can predict boxification lines for each panorama in the same room. Second, we propose a graph-cut based binary segmentation that produces room layouts with sharp corners and straight walls. Third, we also annotated one multi-view consistent layout dataset for this new layout prediction framework. Our quantitative results show an improvement over single-view room layout estimation algorithms.

Manhattan Room Layout Reconstruction from a Single 360∘ Image: A Comparative Study of State-of-the-Art Methods

Chuhang Zou, Jheng-Wei Su, Chi-Han Peng, Alex Colburn, Qi Shan, Peter Wonka, Hung-Kuo Chu, and Derek Hoiem
International Journal of Computer Vision (IJCV), 2021
Paper (Preprint)

Recent approaches for predicting layouts from 360◦ panoramas produce excellent results. These approaches build on a common framework consisting of three steps: a preprocessing step based on edge-based alignment, prediction of layout elements, and a post-processing step by fitting a 3D layout to the layout elements. Until now, it has been difficult to compare the methods due to multiple different design decisions, such as the encoding network (e.g., SegNet or ResNet), type of elements predicted (e.g., corners, wall/floor boundaries, or semantic segmentation), or method of fitting the 3D layout. To address this challenge, we summarize and describe the common framework, the variants, and the impact of the design decisions. For a complete evaluation, we also propose extended annotations for the Matterport3D dataset, and introduce two depth-based evaluation metrics.

Checkerboard Patterns with Black Rectangles

Chi-Han Peng*, Caigui Jiang*, Peter Wonka, Helmut Pottmann
ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH ASIA 2019)
Paper (authors’ version) | Additional Materials | Video

Checkerboard patterns with black rectangles can be derived from quad meshes with orthogonal diagonals. First, we present an initial theoretical analysis of these quad meshes. The analysis reveals many possible applications in geometry processing and also motivates the numerical optimization for aesthetic and functional checkerboard pattern design. Second, we describe an optimization algorithm that transforms initial 2D and 3D quad meshes into quad meshes with orthogonal diagonals. Third, we present a 2D checkerboard pattern design framework based on integer programming inspired by the logo design of the 2020 Olympic games. Our results show a variety of 2D and 3D checkerboard patterns that can be derived from 2D or 3D quad meshes with orthogonal diagonals.

DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama

Shang-Ta Yang, Fu-En Wang, Chi-Han Peng, Peter Wonka, Min Sun, Hung-Kuo Chu
Conference on Computer Vision and Pattern Recognition (CVPR), 2019
Paper (Arxiv)

We present a deep learning framework, called DuLa-Net, to predict Manhattan-world 3D room layouts from a single RGB panorama. To achieve better prediction accuracy, our method leverages two projections of the panorama at once, namely the equirectangular panorama-view and the
perspective ceiling-view, that each contains different clues about the room layouts. Our network architecture consists of two encoder-decoder branches for analyzing each of the two views. In addition, a novel feature fusion structure is proposed to connect two branches, which are then jointly trained to predict the 2D floor plans and layout heights. To learn more complex room layouts, we introduce the Realtor360 dataset that contains panoramas of Manhattan world room layouts with different numbers of corners. Experimental results show that our work outperforms recent state-of-the-art in prediction accuracy and performance, especially in the rooms with non-cuboid layouts.


Designing Patterns using Triangle-Quad Hybrid Meshes

Chi-Han Peng, Helmut Pottmann, Peter Wonka
ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2018)
Paper (authors’ version) | Additional MaterialsVideo | Models

We present a framework to generate mesh patterns that consist of a hybrid of both triangles and quads. Given a 3D surface, the generated patterns fit the surface boundaries and curvatures. Such regular and nearly regular triangl-equad hybrid meshes provide two key advantages: first, novel-looking polygonal patterns achieved by mixing different arrangements of triangles and quads together; second, a finer discretization of angle deficits than utilizing triangles or quads alone. Users have controls over the generated patterns in global and local levels. We demonstrate applications of our approach in architectural geometry and pattern design on surfaces.


Computational Network Design from Functional Specifications

Chi-Han Peng, Yong-Liang Yang, Fan Bao, Daniel Fink, Dong-Ming Yan, Peter Wonka, Niloy J. Mitra
ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2016)
Paper (authors’ version) | Slides

Connectivity and layout of underlying networks largely determine agent behavior and usage in many environments. For example, transportation networks determine the flow of traffic in a neighborhood, whereas building floorplans determine the flow of people in a workspace. Designing such networks from scratch is challenging as even local network changes can have large global effects. We investigate how to computationally create networks starting from only high-level functional specifications. Such specifications can be in the form of network density, travel time versus network length, traffic type, destination location, etc. We propose an integer programming-based approach that guarantees that the resultant networks are valid by fulfilling all the specified hard constraints and that they score favorably in terms of the objective function. We evaluate our algorithm in two different design settings, street layout and floorplans to demonstrate that diverse networks can emerge purely from high-level functional specifications.


Computing Layouts with Deformable Templates

Chi-Han Peng, Yong-Liang Yang, and Peter Wonka
ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2014)
Project Page | Paper (authors’ version) | Additional MaterialsTalk SlidesFast-Forward Slides

In this paper we tackle the problem of tiling a domain with a set of deformable templates. A valid solution to this problem completely covers the domain with templates such that the templates do not overlap. We generalize existing specialized solutions and formulate a general layout problem by modeling important constraints and admissible template deformations. Our main idea for the solution is to break the layout algorithm into two steps: a discrete step to layout the approximate template positions and a continuous step to refine the template shapes. Our approach is suitable for a large class of applications, including floorplanning, urban layouts, and arts and design.


Exploring Quadrangulations

Chi-Han Peng, Michael Barton, Caigui Jiang, and Peter Wonka
ACM Transactions on Graphics (to be presented at SIGGRAPH 2014)
Project PagePaper (authors’ version) | Additional Materials | Video (youtube) | Models | Talk Slides | Fast-Forward Slides

We present a framework for exploring topologically unique quadrangulations of an input shape. First, the input shape is segmented into surface patches. Second, different topologies are enumerated and explored in each patch. This is realized by an efficient subdivision-based quadrangulation algorithm that can exhaustively enumerate all mesh topologies within a patch. To help users navigate the potentially huge collection of variations, we propose tools to preview and arrange the results. Furthermore, the requirement that all patches need to be jointly quadrangulatable is formulated as a linear integer program. Finally, we apply the framework to shape-space exploration, remeshing, and design to underline the importance of topology exploration.


Connectivity Editing for Quad-Dominant Meshes

Chi-Han Peng and Peter Wonka
Eurographics Symposium on Geometry Processing (SGP) 2013
Paper | Additional Materials | Video (youtube) | Slides

We propose a connectivity editing framework for quad-dominant meshes. In our framework the user can edit the mesh connectivity to control the location, type, and number of irregular vertices (with more or less than four neighbors) and irregular faces (non-quads). We provide a theoretical analysis of the problem, discuss what edits are possible and impossible, and describe how to implement an editing framework that realizes all possible editing operations. In the results we show example edits and illustrate advantages and disadvantages of different strategies for quad-dominant mesh design.


Connectivity Editing for Quadrilateral Meshes

Chi-Han Peng, Eugene Zhang, Yoshihiro Kobayashi, and Peter Wonka
ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH ASIA 2011)
Project PagePaper | Additional Materials | Video (youtube) | Slides

We propose new connectivity editing operations for quadrilateral meshes with the unique ability to explicitly control the location, orientation, type, and number of the irregular vertices (valence not equal to four) in the mesh while preserving sharp edges. We provide theoretical analysis on what editing operations are possible and impossible and introduce three fundamental operations to move and re-orient a pair of irregular vertices. We argue that our editing operations are fundamental, because they only change the quad mesh in the smallest possible region and involve the fewest irregular vertices (i.e., two). The irregular vertex movement operations are supplemented by operations for the splitting, merging, canceling, and aligning of irregular vertices. We explain how the proposed highlevel operations are realized through graph-level editing operations such as quad collapses, edge flips, and edge splits. The utility of these mesh editing operations are demonstrated by improving the connectivity of quad meshes generated from state-of-art quadrangulation techniques.


User-Assisted Mesh Simplification

Tan-Chi Ho, Yi-Chun Lin, Jung-Hong Chuang, Chi-Han Peng, Yu-Jung Cheng
VRCIA ’06 Proceedings of the 2006 ACM international conference on Virtual reality continuum and its applications

During the last decade, many simplification methods have been proposed to generate multi-resolution meshes for real-time applications. Practitioners have found that these methods alone usually fail to produce satisfactory result when models of very low polygon count are desired. This is due to the fact that the existing methods take no semantic or functional metric into account, and moreover, each error metric has its own strength and weakness. In this paper, we propose a user-assisted mesh simplification framework that allows users to improve the quality of simplified meshes derived by any error metric. The framework consists of two stages. The first stage employs a weighting scheme that allows users to refine a unsatisfactory region to achieve a user-specified resolution. The second stage is a local refinement scheme aiming to provide a user-guided fine-tune to recover local sharp features. The proposed weighting scheme differs from the previous approaches in that the weights are used to directly reorder the edge collapsing sequence rather than weighting the collapsing cost. Such a direct reordering mechanism ensures a predictable increase of resolution in the selected region, and is both error-metric and resolution independent.


Connectivity Control for Quad-Dominant Meshes with Applications in Urban Design
Chi-Han Peng
Advances in Architectural Geometry (AAG) 2014

Feature Detection in Aerial Images
Cheng Pan, Yifan Zhang, and Chi-Han Peng (Advisers: John Femiani, Anshuman Razdan, and Peter Wonka)
SIAM Data Mining Conference (SDM) 2011 Doctoral Forum