The Embodied Computation Lab is a new facility for researching building systems located on the campus of Princeton University in Princeton, New Jersey. In 2014 The Living was shortlisted to submit a design for the new facility. Our proposal was selected as the winner and finished construction in 2016.

Reclaiming material

Our proposal imagined not just a building that would house research, but that the building itself could become part of the research—less a fixed design and more of an adaptable framework that can be modified and experimented with during its lifespan—a concept we called "the open-source building".

Range of sustainable solutions implemented in the design

While our proposal incorporated many sustainable elements, one of the project's most innovative aspects was the facade, which was composed entirely of scaffolding planks reclaimed from construction sites around New York City. In order to work with the natural variation of the boards (which vary widely in width, thickness, length, and color) and develop a cohesive facade layout, we developed a custom digital imaging pipeline to catalog all boards and created various computational model to efficiently lay them out across the building's facade.

Reveal the structure

Going one step further, we partnered with a local artist working in Brooklyn who was experimenting with sandblasting natural materials including wood to reveal the intricate natural patterns within. We were interested in applying his methods to the entire facade, thereby revealing the beautiful complexity of this often forgotten material and breathing new life into these planks.

A sample of board images after being cleaned and rectified (left), a detail of a section of wood after sandblasting (right)

Our goal was to create a process that could selectively sandblast only the areas of the planks that featured knots, thereby focusing our efforts on the areas that revealed the most hidden variations in the tree's structure. In order to scale the sandblasting process to the entire facade the process including both identifying the areas to sandblast and doing the actual sandblasting had to be largely automated.

Automate through learning

Initially, we considered a deterministic approach using computer vision techniques. However, the intricacy of the problem inspired us to explore more advanced methods, specifically convolutional neural networks (CNNs), which are highly effective for image recognition tasks. By training a neural network to recognize knots, we aimed to teach the system to detect them without explicitly programming it with information about their appearance or location.

Diagram of digital process including image processing, model training, and model prediction

The workflow we developed involved splitting the wooden boards into square patches and manually classifying each patch as containing a knot or not. Importantly, we did not provide any additional information about the knots’ characteristics. To gather sufficient training data, we created a web application that allowed us to crowdsource the classification process among our team and friends. The neural network was trained to identify patterns in the images and create a visual heatmap of the knots, which highlighted the areas where knots were most likely to be found.

'Not or not' - a simple webapp for classifying wood samples

Results of model training showing bootstrapped heatmaps of knot location derived from largest activation in layers of the CNN

After some experimentation with the CNN model's architecture and parameters, the trained model produced remarkably accurate results. The system was able to detect knots with high precision, despite being given minimal input. This demonstrates the power of machine learning algorithms in uncovering structures and visual features in complex materials.

Images showing classification using the final trained model on the reserved test set, with images classified as 'containing knots' on the left and those classified as having 'no knots' on the right. The model shows high accuracy in correctly identifying knot features despite a large variation in color and texture across all samples.

To bring the project full circle, we developed a CNC sandblasting robot that could process the boards based on the neural network's output. The knot detection data was converted into GCODE, enabling the automated processing of all 900 reclaimed boards.

Diagrams of digital process to create machine data and custom robotic CNC sandblasting rig.

The final result was a striking facade for the robotics lab that not only served as a decorative application but also illustrated the potential for machine learning to enhance intelligent design solutions in architecture and the built environment.