Why Testbatch?

Short version

Designing collaboratively between designers and the people who use stuff from concept to manufacturing has long been a challenge. Testbatch is an attempt to apply contemporary digital technologies and design innovation theory to the problem to see if this is a way to design together, better.

Users are already designing a lot of their own things (in fact it’s possible the majority of new product innovation is done by users), and the global economy is at the cusp of a new stage of the industrial revolution where personalisation of objects and digital methods of creation and communication are key (see Industry 4.0, or Design Global Manufacture Local).

Testbatch is an attempt at an interface and process to enable this new future, where the lines between user and designer are blurred, yet each relies on the other more than ever before.

The Pitch

This is how the project has been pitched so far. These slides are from a 10 minute talk on the proposal for Testbatch.

Long Version

Below are adapted parts of the Research Proposal for this project that handed in as part of the research project’s development at the University of Canberra.  All documents submitted as part of the research phase of this project can be accessed here.


Digital technology enables new relationships between user, designer and data that allow for iteration and collaboration quickly, cheaply and clearly. Through the design, iteration and implementation of 3D digital design ‘user toolkits’ the design innovation process can be democratised (von Hippel, 2005), and in so doing create products that better fit user, designer and manufacturer needs. Digital tools allow for the capture of rich quantitative data that can be analysed to generate insight, as well as methods of integrating data into subsequent stages of the design innovation process (Chen, Hoyle & Wassenaar, 2012). This study aims to investigate methods of improving designer decision making with insights generated from user customization data, through the development and use of a digital platform that supports user and designer collaboration (see http://testbatch.com.au) A constructionist epistomelogical approach will be followed, and a hybrid Construction Design Research (CDR) and Action Research methodology implemented in this practice based project.


Research Questions

The core research question is;

How can data generated by user customisation of 3D digital designs democratise the Industrial Design process?

Which leads to two sub-questions that will also be explored;

  1. Can a user generated data based design process enable fairer methods of user/designer collaboration?
  2. How can insights from user generated data be used effectively in the early stages of the industrial design process?

The area of study is defined with this author created Venn Diagram shows what expertise Users, Tools and Designers hold, what methods of creation of collaboration their mixing allows for, and what type of design is being investigated;

Where various disciplines interesect to make this possible
Venn diagram of relevant knowledge domains, by Dean Hewson 2017

Digital technology enables new relationships between user, designer and data that allow for iteration and collaboration quickly, cheaply and clearly. Users have historically been kept at arm’s length in the design process, but through the design, iteration and implementation of 3D digital design ‘user toolkits’ the process can be democratised, and in so doing create products that better fit individuals needs. Design methods like DIY and co-creation are leading the way towards democratisation (Hoftijzer, 2009), and web based design and new manufacturing tools and processes are changing the relationship between novice and expert designers in the innovation process (Blikstein, 2013), and allowing them to connect online effectively.

Digital design tools also allow for the capture of rich quantitative data that can be used to generate insight, as well as methods of integrating that and other consumer preference data into subsequent stages of the design innovation process (Chen, Hoyle & Wassenaar, 2012).

This study sets out to investigate the relationships between designers, users and customisation through the application of a collaborative method of design innovation to develop “User Toolkits” from concept to manufacturability.

See how Design Process works on Testbatch


The design process can be looked at as one of ‘co-evolution’, where the question and answer (or problem space and solution space are evoling together.

Co-evolution Design Model by Maher et al (1996)
Co-evolution Design Model by Maher et al (1996)

A type of design that fits the co-evolution model are user customisable designs. User customisable designs are parametric designs; designs that change based on variations in input data, and have internal relationships that keep the design cohesive in the face of changes (Woodbury, 2010). Designing a system that allows variability creates a solution space of all possible outcomes; the size of the solution space defines the design freedom available (Dorst, 2001). Allowing options only for size or colour creates a small solution space; altering the form of a product creates one with much more design freedom.

When design thinking methods and parametric digital design tools are used in tandem and made available to large groups of people, then that process is on the path to being democratised.

Democratising Innovation

Democratising innovation occurs when users are not just included in the design innovation process, but are active and respected collaborators capable of their own design insight who have been included in an innovation community. The gap between beginners and experts is bridged through the application of ‘User Toolkits’, which will be explained below (Hippel, 2001). Collaborating with user innovators has been shown to be a commercial positive for organisations (Pongtanalert and Ogawa, 2015). Nishikawa, Schreier and Ogawa (2013) found that Muji’s user-generated products generated 4 to 6 times more gross profit than designer-generated products).  In the traditional product design process a user has needs and the manufacturing and design community identify and fill those needs. However, empirical studies have shown that between 10-40% of new innovations brought to market began as user innovations, and the degree to which users contribute is increasing (Hippel, 2005).

User Toolkits

Democratising innovation implements ‘User Toolkits’ as frameworks for open innovation, which enable non-specialist users to design high-quality, manufacturable custom products that exactly meet their needs (Hippel, 2005). User toolkits are interactive systems that allow the designer to set up a parametric design system that contains a solution space of manufacturable products, and the user to create/select their best option from that solution space. This locates the ‘sticky information’ that users have about their needs with the users, and the ‘sticky information’ manufacturers have about how design objects can be produced with the manufacturers (Goduscheit & Jørgensen, 2013). Co-locating design decision making responsibility with this information reduces the number of times information needs to be passed between manufacturer and user, and increases the speed at which learning by doing in a trial and error process is possible (Hippel, 2008).

The users doing the innovation are largely ‘lead users’, meaning that they are early adopters of a market trend, and gain more than most from designing solutions to their needs (Gemser & Perks, 2015). Targeting user design toolkits at lead users helps the manufacturer identify and acquire the useful innovations of the lead users, enabling them to better design their products for the rest of the market (Heineirth, Niedner and Herrmann, 2014). Lag users, that is users who are late adopters, have also been shown to be able to reveal similar design insights as lead users when the user toolkit they engage with has an educational trial and error learning as a focus (Jahanmir & Lages, 2015).

Motivations for using User Toolkits

The 5 core attributes of a high quality user innovation toolkit are that they;

  1. Allow for complete cycles of trial-and-error learning;
  2. Make it possible for users to create the design they actually want;
  3. Require little specialised training;
  4. Allow for other user’s designs to simplify the creation process and;
  5. Ensure a manufacturable outcome.

(von Hippel and Katz, 2002)

The role of the designer in developing a user toolkit and the processes to support it is to enable effective diffusion of insight and understanding between users and manufacturers, and to guide a successful design outcome to completion. A key challenge of user toolkit design is then how to deal with ‘mass confusion’. That is, the burdens the user encounters while engaging in a configuration process (Pine, 1993). The prevalence of mass confusion in user interfaces has been suggested as the reason for the delay in industry-wide adoption of processes for customisable design, but it may be alleviated by embedding innovation in an appropriate ‘collaborative co-design environment’ (Piller, Koch, Moslein and Shubert, 2004). Iterating a user toolkit early with users has large potential for solving this problem. Utterback & Abernathy (1975) also proposed that innovation by users is likely to be most important in the early stages of such a design process. However, most current uses of user toolkits focus on the end of the process, creating the final specifications for manufacturing. Applying new digital and analytical technologies to this problem set appears ready to bear fruit.

For manufacturers, value has been theorised to come from being able to charge premium prices, ‘economies of integration’ born of better access to market information, and customer loyalty (Piller et al., 2004). Customer drivers of value were postulated by Merle, Chandon and Roux (2008), and empirically validated in 2010 as being;

  • Extrinsic
    • Utilitarian
    • Individualism
    • Self expression
  • Intrinsic
    • Hedonism
    • Pride

When these value drivers are met by a user toolkit, then a user’s willingness to pay has been shown to improve (Franke, Kreinz & Schreier, 2008).

Freely sharing knowledge is a strong motivator for some users who engage with open innovation processes. Factors such as reputation and strengthening information communities, as well as diffusing new innovations quickly motivate users (Jerome, 2013). However rates of open innovation among user-innovators appears low, with a recent study indicating only 18% of user-innovators from the USA and 11% of Japanese user-innovators shared their innovations openly, and fewer belong to a related information community at all (Pontangalert & Ogawa, 2015). Whether that low rate is caused by the open innovation model itself, or the open innovation model trying to operate in a closed system remains to be seen.

Despite the many benefits of user toolkits, there are limitations to the standard model (Mayson, 2015). The resulting 3D data is not editable directly by the user, and later users (as well as designers) are faced with difficulties in refining or building from the resulting 3D data of the original users. This deficiency can be addressed by going beyond the ‘product configuration’ type of toolkit, and utlising variable, parametric design principles that allow for rich data capture, storage and transfer.

Industrial Design is concerned with the early stages of the design innovation process, however tools and evaluation methods for analysing users and data that currently exist are focussed on the later stages. Designers need methods of engaging well with and understanding the technologically changed landscape and the effects it has on design innovation, as well as the broader changes to social context in terms of information sharing and notions of ownership. Doing so requires an increase and broadening of domain knowledge held by the designer to include communication and statistical reasoning, and their engagement with the possibilities raised in this literature review by doing projects that utilise them out in the real world.


Aims and Objectives of Testbatch

This project aims to investigate methods of improving designer decision making with insights generated from user customization data.

To support this investigation, a digital platform has begun to be developed (see http://testbatch.com.au) to support digital user and designer collaboration on the development of design outcomes. The platform will have User Toolkits that enable the customization of designs, a project management system for openly and collaboratively developing the toolkits and a capacity to build a community of interested parties around a particular User Toolkit idea.

I will be putting my own designs through the process, and have recruited a small pool of other designers to also openly collaborate on iterating their designs.

Though at all times the toolkits will be developed with the goal of being able to create final physical outcomes, due to time and skill constraints that end goal may not be reached during this research period. Fortunately, the insights generated through facilitating this process with designers and the development of basic guidelines for designing in this way are the primary research? objective. The further outputs for this project will likely be physical objects built using this process or representations thereof, case studies of the user toolkit development, insight into how to design well using this process, and possible directions for future research. These will be presented both as an online and physical exhibition, along with an Exegesis of the project. Hopefully, http://testbatch.com.au will also have a life after the Honours project ends both as a research, teaching and community platform.

See the Data and Privacy page for more information on how data is used on Testbatch


Potential Significance

Democratising innovation as an outcome and as a process has the potential to be a significant engine of economic and social change. As it becomes easier for users to get exactly what they want by designing it themselves, an increase in social welfare has been theorised (Gambardella, Raasch and von Hippel, 2016). Manufacturers are however not necessarily concerned with social welfare, and open innovation requires supporting open knowledge transference as much as possible (Bogers, McCarthy and Pitt, 2015) which reduces the opportunities for manufacturers to profit from exclusivity. Demonstrating the ease with which product research and community development can occur through http://testbatch.com.au may allay those fears.

Further motivations and key features of a user toolkit are the capacity for guiding learning and the joy derived from engaging with the toolkit (Hertel, Niedner and Herrmann, 2003; Lakhani & Wolf, 2005). Given that lag users (ie, late adopters) can generate insight of similar quality to lead users if the user toolkit includes extra explanatory information (Jahanmir & Lages, 2015), and non-designers have been seen in case studies to outperform pure design teams (von Hippel, 2005), then the potential for improving the market fit, usability and aesthetics of a product is clear.

The new economic arrangements an online collaborative system built around the codesign of user tookits between users and designers could enable further highlight this project’s significance. A democratised innovation system may fit well in the ‘design global – manufacture local’ post-capitalist paradigm suggested by Kostakis et al (2015), as a global community could work on the designs, but each customised outcome has to be made in one location. One potential future iteration of http://testbatch.com.au  could allow manufacturers to bid on the manufacturing and shipping of the customised product to the user.

The information and communication structure of http://testbatch.com.au may also offer itself to a Platform Cooperativist ownership model, where the stakeholders in a platform also own the platform (Scholz and Schneider, 2016). Finding a fair method to remunerate people whose creative and technical labour is used to develop designs on http://testbatch.com.au is a secondary research question of this project and likely core to building the trust and motivation needed between the users and designers on the platform to be successful.

Reference List

Blikstein, P (2013). Digital Fabrication and ‘Making’ in Education: The Democratization of Invention, in FabLabs: Of Machines, Makers and Inventors, J. Walter-Herrman and Buching, C., Editors, Bielefeld

Bogers, M., McCarthy, I. P., & Pitt, L. (2015). Leveraging users as innovators: Managing the creative potential of individual consumers. Journal of Engineering and Technology Management, 37, 3–5. https://doi.org/10.1016/j.jengtecman.2015.09.001

Brown, T. and Katz, B. (2009). Change by design: how design thinking transforms organizations and inspires innovation. Harper Business.

Chen, W., Hoyle, C., & Wassenaar, H. J. (2012). Decision-based design: Integrating consumer preferences into engineering design. Springer Science & Business Media.

Council, Design. “The ‘double diamond’design process model.” Design Council (2005).

Cross, N. (1999). Design Research: A Disciplined Conversation. Design issues, 15(2), 5.

Cross, N. (2001). Designerly Ways of Knowing: Design Discipline Versus Design Science. Design issues, 17(3), 49-55

Dorst, K. (2001). Creativity in the design process: co-evolution of problem–solution. Design Studies, 17(4), 341–361.

Franke, N., Keinz, P., & Schreier, M. (2008). Complementing mass customization toolkits with user communities: How peer input improves customer self-design. Journal of Product Innovation Management, 25(6), 546–559. https://doi.org/10.1111/j.1540-5885.2008.00321.x

Friedman, K. (2003). Theory construction in design research: criteria: approaches, and methods. Design Studies, 24(6), 507-522.

Gambardella, A., Raasch, C., & von Hippel, E. (2016). The User Innovation Paradigm: Impacts on Markets and Welfare. Management Science, (July), mnsc.2015.2393. https://doi.org/10.1287/mnsc.2015.2393

Gemser, G., & Perks, H. (2015). Co-Creation with Customers: An Evolving Innovation Research Field. Journal of Product Innovation Management, 32(5), 660–665. https://doi.org/10.1111/jpim.12279

Goduscheit, R. C., & Jørgensen, J. H. (2013). User toolkits for innovation – a literature review. International Journal Technology Management, 614(34), 274–292. https://doi.org/10.1504/IJTM.2013.052671

Hearn, G. N., Tacchi, J. A., Foth, M., & Lennie, J. (2009). Action research and new media: Concepts, methods and cases. Hampton Press.

Hertel, G., Niedner, S., & Herrmann, S. (2003). Motivation of software developers in Open Source projects: an Internet-based survey of contributors to the Linux kernel. Research Policy, 32(7), 1159–1177. https://doi.org/10.1016/S0048-7333(03)00047-7

Hoftijzer, J (2009). DIY and Co-creation: Representatives of a Democratizing Tendency. Design Principles and Practices: An International Journal. 3(6): p. 69- 82

Jahanmir, S.F. & Lages, L.F., 2015. The Lag-User Method: Using laggards as a source of innovative ideas. Journal of Engineering and Technology Management, 37, pp.65–77.

Jerome, L. W. (2013). Innovation in social networks: Knowledge spillover is not enough. Knowledge Management Research and Practice, 11(4), 422–431. https://doi.org/10.1057/kmrp.2012.44

Jupp, V. (2006). The SAGE Dictionary of Social Research Methods. (V. Jupp, Ed.). London, England, United Kingdom: SAGE Publications, Ltd. https://doi.org/10.4135/9780857020116

Koskinen, I., Zimmerman, J., Binder, T., Redstrom, J., & Wensveen, S. (2011). Design Research Through Practice: From the Lab, Field, and Showroom. Elseveir.

Kostakis, V., Niaros, V., Dafermos, G., & Bauwens, M. (2015). Design global, manufacture local: Exploring the contours of an emerging productive model. Futures, 73, 126–135. https://doi.org/10.1016/j.futures.2015.09.001

Lakhani, K. R., & Wolf, R. G. (2005). Why Hackers Do What They Do: Understanding Motivation and Effort in Free / Open Source Software Projects, Perspectives on Free and Open Source Software, 1–27. https://doi.org/10.2139/ssrn.443040

Maher, M L, Poon, J and Boulanger, S ‘Formalising design exploration as co-evolution: a combined gene approach’, in J S Gero and F Sudweeks (eds) Advances in formal design methods for CAD, Chapman and Hall, London, UK (1996)

Merle, A., Chandon, J.-L., & Roux, E. (2008). Understanding the perceived value of mass customization: the distinction between product value and experiential value of co-design. Recherche et Applications En Marketing, 23(3).

Nishikawa, H., Schreier, M., & Ogawa, S. (2013). User-generated versus designer-generated products: A performance assessment at Muji. International Journal of Research in Marketing, 30(2), 160–167. https://doi.org/10.1016/j.ijresmar.2012.09.002

Norman, D. (2013). The Design of Everyday Things. Human Factors and Ergonomics in Manufacturing (Vol. 16). Tantor Media, Inc. https://doi.org/10.1002/hfm.20127

Piller, F., Koch, M., Möslein, K. M., & Schubert, P. (2004). From Mass Customization To Collaborative Customer Co-Design. ECIS, (2004), 1497–1509.

Pine, B. J. (1993). Mass customization: the new frontier in business competition. Harvard Business School Press.

Pongtanalert, K., & Ogawa, S. (2015). Classifying user-innovators – An approach to utilize user-innovator asset. Journal of Engineering and Technology Management – JET-M, 37, 32–39. https://doi.org/10.1016/j.jengtecman.2015.08.005

Utterback, J., & Abernathy, W. (1975) A Dynamic Model of Process and Product Innovation, International Journal of Management Sciences. 3(6) pp 639–656

Scholz, T., & Schneider, N. (2016). Ours to Hack and to Own: The Rise of Platform Cooperativism, a New Vision for the Future of Work and a Fairer Internet. OR Books

von Hippel, E.  (1998). Economics of Product Development by Users: Impact of “Sticky” Local Information. Management Science, 44(5), 629–644. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=

von Hippel, E. (2001). PERSPECTIVE: User toolkits for innovation. Journal of Product Innovation Management, 18(4), 247–257. http://onlinelibrary.wiley.com/doi/10.1111/1540-5885.1840247/abstract

von Hippel, E., & Katz, R. (2002). Shifting Innovation to Users via Toolkits. Management Science, 48(7), 821–833. https://doi.org/10.1287/mnsc.48.7.821.2817

von Hippel, E. (2005). Democratizing innovation. Journal Für Betriebswirtschaft. Cambridge, MA: MIT Press. https://doi.org/10.1111/j.1540-5885.2006.00192_2.x
Woodbury, R. (2010). Elements of Parametric Design. Taylor and Francis.

World Design Organization. (2016). Definition of Industrial Design. Retrieved March 13, 2017, from http://wdo.org/about/definition/