International Journal of Contents

The Korea Contents Association (한국콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

An Exploratory Study on the Usage Patterns of Software-based Design Tools in Designers' Ideation and Collaboration Activities

  • Received : 2021.12.15
  • Accepted : 2021.12.27
  • Published : 2021.12.28
Download PDF
⟨ Previous Next ⟩

Abstract

The purpose of this study was to explore how designers use software-based design tools for ideation and collaboration (for two cases: with designers and with developers). We conducted logistic regression analysis and random forest analysis. Software-based design tools are more popular among product designers and affiliated with design organizations with 51 to 100 members. We identify the features that influence designers to use design tools for the ideation and collaboration, and how these usage patterns are interrelated. Interrelated usage pattern is a key consideration for location of the menu and convenience of use. The results imply that reinforcement of the design tool features per designer profile is required and that design management should be consistent with the field of design and the nature of the organization.

Keywords

1. Introduction

Software-based design tools have become indispensable for designers. It is now virtually impossible to carry out basic design work without such tools [1]. There are a variety of design tools specializing in different areas and with different strengths. Depending on the field of design, the tools are classified as specializing in graphic design [2,3] and in product design [4]. The tools may differ depending on the nature of the organization. Hence, the characteristic of the employer and the size of the organization may affect the designers’ preference for design tools. Nevertheless, all design tools are developed to address the challenge of supporting average level designers in delivering the final deliverable, which complies with the customer’s quality requirements in the quickest manner possible.

Using the new features of these design tools, designers can now easily deliver high-quality deliverables in a relatively short time [5]. This means that the designers’ technical capabilities have increased with the improvement in the implementation and caliber of the tools. In this environment, designer’s creative ideation abilities and capability to reflect customer needs in design are important criteria for appraising them. Furthermore, organizational leaders are now interested in intra-organization communication ability as a key capability of designers for enhancing the quality of the final deliverables [6].

Therefore, design tool developers must now develop tools that support creative ideation and promotion of inter/intra-organizational collaboration via communication, ultimately leading to higher quality design.

Design tools have different scopes and uses, different characteristics and backgrounds. However, designers need to complete some essential activities to identify customer needs and complete deliverables. From the perspective of design as a means to complete innovation, these activities can be further categorized into bringing creativity to reality and reinforcing execution [7,8]. Among the abovementioned three capabilities, ideation is deeply related to bringing creativity to reality [9,10] while the other two (collaboration with designers and with developers that finally implement design deliverables into final products/services) are key capabilities of designers to reinforce execution ability [11,12].

This paper explores the relationship between design tools and the three essential activities: ideation, collaboration between designers, and collaboration between designers and developers. The research questions addressed are as follows:

1. Do designers use design tool for the above three activities? If so, what type of designers actively utilize such tools?

2. What are other features and the usage patterns of the design tools affect the three essential activities?

3. How does designers’ rating of design tools affect the three essential activities?

4. Are these essential activities interrelated from the perspective of design tool use?

2. Literature review

2.1 Design tool

Literature on design tools has progressed in two directions. One presents the features and benefits of a design tool specializing in a particular field, and the other highlights the positive results of the design education using the tool. [13]. Some analyze user accessibility for selected prototyping tools (Sketch, Adobe XD, Balsamiq, UXPin) [14], and some identify visual factor categories in video editing and design [15]. In motion graphics, some analyze elements from the perspective of forms and communication [16], and others examine the classification systems of motions to propose the future evolution of design tools [17]. In architecture, the role and the benefits of simulation tools [18], 3D CAD, and VR-based design interfaces are explored [19].

While some studies have emphasized the usefulness of 3D modeling tools, others have point out their limitations. Other design tools (e.g., SketchUp and Silhouette Modeler) are presented as alternatives to 3D CAD, while staged approaches for 3D design and interface enhancement are also proposed. These essentially attempt to improve traditional design work by identifying more optimal design tools for respective fields [20].

Second, among the design processes, many use design tools as an ideation tool in the stages of ideation, concept generation, and sketching [21]. Ideation can be further broken down into brainstorming at the early stages and then sketching [22].

Some study how software-based design tools are used in brainstorming [23], where such tools, rather than paper-based sketches, help designer ideate [24], and where such tools are not helpful [1]. In the ideation stage, the contribution of design tools is evaluated based on how creative ideas are proposed and detailed [25].

Third, the collaboration features of design tools draw attention. This is not limited to design. Other fields have embraced collaboration tools to enhance productivity. As design collaboration directly affects an organization’s deliverables, research on the collaboration features is indispensable.

Designers may collaborate with an average citizen [26], other designers or colleagues in other value chains (e.g., marketing and R&D) [27], and experts in other disciplines [28]. Collaboration using such tools enhances design deliverables and completes innovation by sharing the design objectives [27], enabling real-time collaboration, reducing work time by digitalizing physical tools [29,30], and overcoming the constraints of space and time [31].

The evaluation of design tools (strengths and weaknesses) and users’ satisfaction depends on the type of work occurring in the design organization and the project characteristics. This is because the selection of and fit of design tools depend on the type of company and work characteristics. For example, photoshop is preferred for large projects, while Sketch tools, which are agile and faster, are preferred by IT service companies and startups [32].

Organizations that use design tools believe that they contribute to better design and organizational performance. Such organizations consider the following: processes which require design tools, the tools appropriate for different projects, and those helpful for designers.

Design tools are significantly related to designers’ mindset, behavior, and other variables within the organization. There is a relationship among ideation in the early stage, designer’s creativity, and inter/intra- organization collaboration.

2.2 Design tool and ideation

Design ideation involves various methods and tools. Experienced designers tend to use accumulated idea archives, but novices/students often lack these assets [33]. However, experience does not always guarantee design creativity. Nevertheless, all designers endeavor to maximize design creativity and software-based design tool is one of the means for this. Chung and Lee (2013) demonstrated that a particular feature of a 3D modeling tool affected the designers’ ideation and concept sketch, where the intensity of the effect was related to the tool’s feature and the designer’s familiarity its usage. Ideation ability, from sketch to concept generation, is evaluated based on the criteria of novelty, completeness, and variety [34]. Among these, novelty is key in design creativity and is often used as the ultimate criterion in evaluating design ideation [35].

A major debate in this field is whether such tools enhance creativity [24]. The tools provide a choice: spend a relatively long time to design on paper or quickly apply examples of successful products using the tools. The choice, often by the design organization, depends on the organizational characteristics/objectives and customer needs.

Creativity is a recurring theme across the design process. There is a study on how creativity impacts the later stages of the design process[36], but creativity is the key success factor for projects and designers in the design ideation stage [37].

2.3 Design tool and creativity

Indeed, designer’s creativity has long been researched in the field of design, and studies that consider the direct relationship between design tools and creativity is emerging. Such studies define the design tool to not only include the design tool kit implemented in the form of software but also differentiated processes and methodologies to achieve a particular task [25].

Researchers have highlighted the importance of creativity and explored impediments to creativity with surveys [38]. Dorst and Cross (2001) also applied a problem-solution perspective to examine the position of creativity in design [39].

Some studies consider the direct relationship between design tool usage and designer creativity. Using CAD can potentially enhance design creativity [40] and there are benefits to using digital 3D in concept design [41]. Although CAD benefits by converging design and machine design, there are apparent limitations[42]. For example, it can limit creativity in designing electronic products [43] and impede diversity of design [31]

Scholars studying creativity based on the overall innovation process, including design, claimed the relationship between creativity and collaboration from the perspective of open innovation [44]. In terms of collaboration, there are various of actors with different roles and design tools can enhance creativity in this setting, highlighting the importance of the relationship between creativity and collaboration [45]. These claims imply that designer creativity and collaboration within organizations are important variables and interrelated.

2.4 Design tool and collaboration

2.4.1 Collaboration between designers

Co-design is a challenging collaboration process in which participants share knowledge, imagination and power for better design outcomes [46,47]. Through this collaboration, designers are influenced by other participants' decisions and given new roles to make better choices [48]. The entities in the cooperation are not confined to designers only and can include developers as well. This paper focuses on the two axes of cooperation: the communication and interaction between designers and between designers and developers.

Traditionally, collaboration between designers has been considered as one of the success factors of a project. Therefore, emphasis was placed on promoting communication in design processes and developing collaborative design systems and processes to reduce interface errors between designers. It was also deemed important that barriers to collaboration be identified on the three levels of actors, projects, and firms, along with the means to eliminate the barriers. Furthermore, face-to-face communication, and managerial and organizational aspects of the project were emphasized as important factors to foster collaboration [49]. Designers even attempted to use notepads as a tools to support co-design [50]. This meant that designers recognized the importance of collaboration between designers as members of the organization, and that the designers are endeavoring to systemize the collaboration as leaders. Indeed, designers are beginning to look for an easier methodology to implement in their frequently used design work tools.

2.4.2 Collaboration between designers and developers

Where the designer’s deliverables are implemented in the form of products/services, the communication between designers and developers is indispensable. However, the two job groups have different tendencies, experiences, and skills, thus generating possible discord in processes and communication. Therefore, collaboration between designers and developers needs to be optimized, which can determine the success or the failure of a project. Collaboration needs to start in the early stages of the project. It is also beneficial to co-locate the project participants [51]. Misunderstandings can unintentionally cause inconsistency in implementation. To eliminate inefficiency of rework in such circumstances, it may be necessary to adopt a conversational system [52]. Indeed, the importance of design tools as a collaboration tools for both designers and software developers has been emphasized, and there are attempts to support diverse features, develop optimized interfaces, and develop plug-ins for this purpose [53,54].

The literature reviewed thus far is summarized graphically in Figure 1. Design tools are used for ideation, and ideation is the most important design process where creativity is realized. The use of design tools can impact creativity both positively and negatively. Design tools are also used for collaboration, where facilitation of collaboration with design tools can positively impact the creativity of the deliverables. A designer realizing creativity and enhancing execution via collaboration can lead to the successful implementation of an innovation.

E1CTBR_2021_v17n4_16_f0001.png 이미지

Figure 1. Overarching framework deduced from the literature review

3. Research Methodology

3.1 Data variables

Data from the 2020 Design Tools Survey is analyzed with permission from the data holder. The survey is conducted by UXtools, web site of design experts, and surveyed designers subscribed to design newsletters, where questions regarding the usage patterns of software-based design tools were asked. Since the respondents are socially active, we filtered the respondents twice to compensate for potential errors and bias. First, we excluded incomplete responses to the survey, and second, we excluded student-designers and design tool users who were not designers. As a result, 4, 100 responses were filtered to 3, 706 responses. The most popular roles were product designers (1, 477 respondents, 39.9%) and UX designers (1, 420 respondents, 39.8%). 80% of total respondents had three or more years of experience. In the case of designers who are affiliated with organizations, the most frequent number of designers in the particular organization was between two and ten. Furthermore, most designers covered two fields of mobile and web (69%) while only 31% covered only mobile or web.

Among the respondents’ profiles, we selected the following features as one of the independent variables: the role of the designer, the number of years as a designer, the number of designers in the affiliated organization, and the field of work (mobile/web) (see Table 1).

Table 1. Profile of respondents

E1CTBR_2021_v17n4_16_t0001.png 이미지

We also set three binary dependent variables to answer the research questions in the introduction: 1) Are software-based design tools used for brainstorming or ideation? 2) Are they used as the design management system for collaboration between designers? 3) Are they used for collaboration between designers and developers? Furthermore, we selected as independent variables whether a design tool is used for a specific purpose and designers’ rating on using the tool for the specific purpose. The four specific purposes are user flows & site map, interface design, UI prototyping, and user testing. We also constructed three research models, taking different dependent and independent variables, as shown in Table 2. These research models are constructed to exclude variables from being independent variable if selected as dependent variable. When a design tool is used for a specific purpose, the corresponding rating of designers was also excluded from the variable. (e.g., if brainstorming & ideation is the dependent variable, both the factor and designers’ rating of design tools for brainstorming & ideation are excluded from the independent variables).

Table 2. Variables considered and three research models

E1CTBR_2021_v17n4_16_t0002.png 이미지

1 mds: managing design system for collaboration between designers

2 handoff: handoff includes collaborating with developers to communicate flows, interactions, assets, and other specifications

3 rating. brainstorming & ideation: rating of design tools for brainstorming and ideation

Table 3. Variables and questionnaires

E1CTBR_2021_v17n4_16_t0003.png 이미지

3.2 Exploratory data analysis

First, we conducted EDA to examine the pattern and distribution of the data [55]. EDA includes graphical tools such as box plots and stem and leaf plots, and analysis, such as correlation [56]. EDA is detective work that finds data patterns [57], which help create hypotheses and refine research questions[55]. Whether to use a design tool for a specific purpose was measured as a binary variable, and the rating of the design tool for the specific purpose was measured on an ordinal scale. We used R to create histograms and box plots to conduct exploratory data analysis. Furthermore, the correlation analysis among variables was conducted to identify the overall data characteristics

3.3 Random forest ans the classifier

We adopted random forest and logistic regression methodologies for classification modeling to explore the second and third research question: what are the features (other than the ones used for the three purposes) and usage patterns of design tools and rating of design tools that influence the use of design tools for the three purposes (ideation, collaboration between designers, and collaboration between designers and developers)? Thus, we are interested in identifying how important features, other than those used for the three purposes, are in the binary classification of the dependent variable.

We used random forest prediction modeling with designer profiles, designers’ design tool usage behaviors, and designers’ ratings of design tool as independent variables, predicting the three dependent variables. Random forest is considered to be one of the most accurate machine learning classification models, even if there is a constraint of data size. It also derives importance of the features (independent variables). Random forest is a machine learning methodology that uses multiple decision trees. A classification tree is fitted to each bootstrap sample derived from the data using random sampling with replacement. At each node, features are randomly selected without overlap. Ultimately, the trees are fully grown, and each is used to make predictions by majority vote. The majority vote means that the prediction taken by the larger number of decision trees is selected if different decision trees yield different predictions. Random forest is an ensemble model consisting of multiple decision tree models [58,59].

3.4 Binary logistic analysis

Logistic regression tends to be less accurate in prediction but can identify the statistical significance of independent variables. Therefore, we additionally used binary logistic classification model, which classifies or predicts the nominal dependent variable. In this research, the independent variables are nominal or ordinal while the dependent variables are binary nominal, which is consistent with the logistic model. Unlike general linear regression, the statistical model is tested using chi-squared tests, while the coefficients are tested using Wald statistics. Furthermore, pseudo R2 such as Cox and Snell R2 and Ngelkerke R2 are used to evaluate the goodness of fit of the model instead of R2.

Binary logistic analysis uses the concept of odds to interpret the results, where the odds is the ratio of the probability of success (pi) over the probability of failure (1-pi): pi /1-pi. This, in turn, leads to binary logistic model equation as follows, where the coefficients measure the change of odds ratio due to each independent variable:

\(\ln \left(\frac{p}{1-\mathrm{p}}\right)=\beta 0+\beta 1 \mathrm{X} 1+\beta 2 \mathrm{X} 2+\cdots+\beta \mathrm{nXn}\)       (1)

Hence, we used logistic analysis to explore statistically significant independent variables that can predict dependent variables [60].

4. Result

4.1 Descriptive statistics and exploratory data analysis (EDA)

In total, 64% of the respondents used design tools for brainstorming & ideation, while 81% used design tools for collaboration between designers and for collaboration between designers and developers.

Of the designers who used design tools for brainstorming & ideation, designers with roles in graphics (45%) and web (42%) were less likely to use design tools than the average (64%). However, those in product design (69%), product management (73%), and UX (66%) were more likely to use design tools. Further, more experienced designers were likely to use design tools. When the affiliated organization had 51~100 designers, the designers were most likely to use design tools (84%). Finally, designers in the mobile field (54%) were less likely to use design tools than those in web (62%) and mobile & web (62%).

In the case of designers using design tools for collaboration (between designers and between designers and developers), designers were most likely to use design tools when they work in product and UX design, had three or more years of experience, or were affiliated with organizations with 51~100 designers. Furthermore, designers in the field of mobile (82%) were more likely to use design tools than those in the field of web (77%) while collaborating with other designers. In the case of collaboration between designers and developers, designers working in both fields of mobile and web (83%) were more likely to use design tools than the other two categories (web or mobile alone).

E1CTBR_2021_v17n4_16_f0002.png 이미지

Figure 2. Descriptive statistics

The ratings of three dependent variables and the four features show interesting patterns. In terms of brainstorming and ideation, a means to realize creativity, many designers rate it as zero because they do not use design tools at all (Figure 3. 1-1). In the case of collaboration between designers (Figure. 3.2-2) and collaboration between designers and developers (Figure 3. 3-2), many designers use design tools and rate them highly, and both categories seem to be distributed in a similar manner. Designers who use design tools to collaborate with developers, tend to give ratings of more than two and rarely give ratings of one (Figure 3. 3- 1). When the design tools are used for user flows & sitemap, interface design, and UI prototyping, the ratings of designers for design tool is relatively high, while user testing suggests otherwise (Figure 3. 7-1).

E1CTBR_2021_v17n4_16_f0003.png 이미지

Figure 3. Histogram and Box plot for EDA

The correlation analysis suggests that the ratings of three dependent variables and the ratings of four features are statistically significant. The ratings of brainstorming & ideation and the ratings of user flow & sitemap had relatively high correlation (0.337). Furthermore, the ratings of collaboration between designers and the ratings of collaboration between designers and developers also exhibited relatively high correlation (0.298)

Table 4. Correlations between variables

E1CTBR_2021_v17n4_16_t0004.png 이미지

rating1: rating of design tool

** Correlation is significant at the 0.01 level (2- tailed)

4.2 Classification result

4.2.1 Random Forest

When predicting design tool usage for brainstorming and ideation, the accuracy of the model was relatively low (69%), while predicting design tool usage for collaboration between designers or between designers and developers had higher accuracy (83.8% and 83.7%, respectively). Since it is not the objective of this research to enhance accuracy, we did not engage in feature engineering to improve accuracy.

In this study, the importance of each feature is more significant. Overall, ordinal variables exhibited more importance than binary variables. For predicting design tool usage for brainstorming & ideation, the ratings of user flows & site maps were the most important feature, while the design tool usage for both collaborations (between designers and between designers and developers) were also important. Second, design tool usage for collaboration between designers and developers and its ratings were deemed important features in predicting the design tool usage in a collaboration between designers. For predicting design tool usage in a collaboration between designers and developers, design tool usage for collaboration between designers (along with its ratings) and the ratings of UI prototyping were considered important features.

Table 5. Accuracy and importance of features in Random Forest modelE1CTBR_2021_v17n4_16_t0005.png 이미지

4.2.2 Logistic regression

While it was possible to derive important features for predicting dependent variables using random forest, it was impossible to obtain the statistical significance of the features and the extent of their impacts (regression coefficients). To address these challenges, logistic regression analysis was conducted. Similar to random forest modeling, we set three models predicting one dependent variable at a time, a chi square test indicated that the models were statistically significant. Hosmer and Lemeshow test, which validated the difference between observed and predicted values, also displayed statistical significance. The pseudo R2 values were derived as shown in Table 6: 21%, 16.7%, and 24.1%. The pseudo R2 is relatively less important than its counterpart in linear regression, while accuracy is more important in determining whether the model is appropriate. The accuracy is similar to that of random forest modeling.

Table 6. Accuracy and pseudo R2 in Logistic model

E1CTBR_2021_v17n4_16_t0006.png 이미지

To maximize the accuracy of the three models (Y1, Y2, Y3), we derived the optimal cut- off value and corresponding accuracy. The optimum cut-off values are 0.49 (Y1), 0.48 (Y2), and 0.35 (Y3) (Figure 4).

E1CTBR_2021_v17n4_16_f0004.png 이미지

Figure 4. Classification cut off curve of Model accuracy

This model’s accuracy refers to the measure that is used when the distribution of each label of binary variables is the same. However, the model’s performance in the case of imbalanced label distribution is distorted and needs to be complemented. To do so, we derived a receiver operation characteristic (ROC) curve and calculated the area under the curve (AUC) to determine if the model is fit. Accuracy is calculated using true positive and true negative. On the other hand, ROC shows how TPR (True Positive Rate: Y axis) changes as FPR (False Positive Rate: X axis) changes, where FPR denotes the ratio of false prediction to true negative. AUC means the area under the ROC curve. As the curve deviates from the diagonal, AUC converges to 1 and the model can be deemed perfect. For values of AUC between 0.8 and 0.9, the model is considered to be excellent and acceptable if AUC is over 0.7; this is not an excellent value but the model can be used [61,62].

\(\begin{aligned} &\text { Confusion Matrix }\\ &\begin{array}{llll} \hline & & \begin{array}{l} \text { Predicted Class } \\ \text { Negative (0) } \end{array} & \text { Positive (1) } \\ \hline \text { Actual } & \text { Negative } & \text { TN } & \text { FP } \\ \text { Class } & (0) & \text { (True Negative) } & \text { (False Positive) } \\ { 2 - 4 } & \text { Positive } & \text { FN } & \text { TP } \\ & (1) & \text { (False Negative) } & \text { (True Positive) } \\ \hline \end{array} \end{aligned}\)       (2)

\(\text { Acurracy }=\frac{\mathrm{TN}+\mathrm{TP}}{(\mathrm{TN}+\mathrm{FP}+\mathrm{FN}+\mathrm{TP})}\)       (3)

\(\text { Precision }=\frac{\mathrm{TP}}{(\mathrm{FP}+\mathrm{TP})}\)       (4)

\(\text { Recall( }=\text { True Positive rate) }=\frac{\mathrm{TP}}{(\mathrm{FN}+\mathrm{TP})}\)       (5)

\(\text { Specificity }(=\text { True negative rate })=\frac{\mathrm{TN}}{(\mathrm{TN}+\mathrm{FP})}\)       (6)

\(\text { False Positive rate }(=100-\text { Specificity })=\frac{F P}{(T N+F P)}\)       (7)

E1CTBR_2021_v17n4_16_f0005.png 이미지

Figure 5. ROC curve and AUC

The features deemed statistically significant in predicting dependent variables (p-value of less than 0.05 and hence rejection of null hypothesis) are listed in Table 7. Logistic analysis is interpreted based on the results of Exp(B) (odds ratio) and the odds ratio of categorical variables against the reference category (as there is a categorical variable). The reference category was graphic designer, and the results imply that the product managers and product designers were more likely to use design tools for brainstorming and ideation than graphic designers by a factor of 1.86 and 1.44, respectively. As the number of designers in the affiliated organization increased, designers were more likely to use design tools for brainstorming and ideation, while designers in both the mobile and web fields are more likely to use design tools for brainstorming and ideation than designers in the field of web only by a factor of 1.52. Furthermore, an increment of one in the rating of user flows and sitemaps indicated that the designers were more likely to use design tools for brainstorming and ideation by a factor of 1.378, while using design tools for user testing meant a higher probability of design usage for brainstorming and ideation by a factor of 1.86 when compared to designers not using the tools for user testing (Table 7). Figure 6 (a) displays the odds ratio of Table 7 in confidence intervals to depict statistically significant variables. When the 95% confidence interval of odds ratio does not include the value of 1, the variable is statistically significant. Figure 6 (b) shows that the increase in the rating of user flows and sitemaps accompanies an increase in the probability of using design tools for brainstorming and ideation.

Table 7. Logistic regression: Design tool usage for brainstorming and ideation

E1CTBR_2021_v17n4_16_t0007.png 이미지

E1CTBR_2021_v17n4_16_f0010.png 이미지

Figure 6. (a) Plot for Odds Ratios, (b) Effect of design tool rating for user flows & sitemaps on the probability of using design tool for brainstorming and ideation

There were four statistically significant features for explaining the dependent variable of design tool usage for collaboration between designers. Product designers were more likely to use design tools for collaboration between designers than graphic designers by a factor of 1.77. Meanwhile, designers in affiliated organizations with 51~100 designers were more likely to use the tools for collaboration between designers than designers in affiliated organizations with one designer by a factor of 3.575. Furthermore, increment in rating of interface design leads to higher probability of design tool usage for collaboration between designers by a factor of 1.154. Figure 7 depicts odds ratios to indicate statistically significant variables, while Figure 8 depicts how increase in rating of interface design and increase in the rating of design tool usage for collaboration between designers and developers lead to higher probability of using design tools for collaboration between designers.

Table 8. Logistic regression: Design tool usage for collaboration between designers

E1CTBR_2021_v17n4_16_t0008.png 이미지

E1CTBR_2021_v17n4_16_f0006.png 이미지

Figure 7. Plot for Odds Ratios

E1CTBR_2021_v17n4_16_f0007.png 이미지

Figure 8. Effect of (a) and (b) on the probability of using design tools for collaboration between designers

(a) Rating of the design tool used for interface design (b)Rating of the design tool used for developer handoff

Finally, we identify that web designers and product designers are more likely to use design tools for collaboration between designers and developers than graphic designers by a factor of 1.56 and 3, respectively. In terms of experience, designers are more likely to use design tools for collaboration between designers and developers than designers with experience of less than one year as follows: 1.5x (1~2yrs), 2.18x (3~5yrs), 2.7x (6~10yrs), 2.6 (more than 10 years). When the designer is affiliated with an organization with two to ten designers, he/she is more likely to use design tools for collaboration between designers and developers than the designer in an organization with one designer by a factor of 1.59, while designers in both the mobile and web fields are more likely to use design tools for collaboration between designers and developers than designers in the field of web only by a factor of 1.45. Designers using design tools for UI prototyping are more likely to use design tools for that purpose by a factor of 3.28 than the designers that do not use design tools for UI prototyping. Designers using design tools for collaboration between designers are likely to use design tools for collaboration between designers and developers by a factor of 3.89, more than the designers that do not use design tools for collaboration between designers. Finally, as the rating of interface design increments, the probability of using design tools for collaboration between designers and developers increases by a factor of 1.24 (Table 9). Figure 9 depicts the odds ratios indicating statistically significant variables, while Figure 10 shows how the probability of design tool usage for collaboration between designer and developer increases as rating of interface design increases.

Table 9. Logistic regression: Design tool usage for collaboration between designer and developer

E1CTBR_2021_v17n4_16_t0009.png 이미지

E1CTBR_2021_v17n4_16_f0008.png 이미지

Figure 9. Plot for Odds ratios

E1CTBR_2021_v17n4_16_f0009.png 이미지

Figure 10. Effect of design tool rating for Interface design on the probability of using design tools in developer collaboration

5. Discussion and conclusion

This study, based on a software-based design tool usage survey, analyzed how design tools are used for brainstorming & ideation, collaboration between designers, and collaboration between designers and developers. The rationale behind the three variables is that innovation becomes complete as creativity and execution are realized [7,8]. Therefore, the activities of designers that are deemed to resemble the theme of creativity and execution are selected. Brainstorming and ideation is a variable that expresses creativity [9,10], while collaboration between designers and collaboration between designers and developers are variables expressing the execution aspect of innovation [11,12]. We aimed to identify the features that influence designers to use design tools for the three purposes and how these usage patterns are interrelated. The explanatory variables can be designer profiles or use of design tools for other purposes. By identifying these variables, we deduced implications such that it is possible to enhance specific features of design tools for different designer profiles and to improve the user interface.

The explanatory variables and implications from the results can be summarized as follows. First, we identify that designers using design tools for ideation use the same design tools for user testing and user flow & site maps. Furthermore, the three dependent variables (design tool usage for ideation, collaboration between designers, and collaboration between designers and developers) all proceed in the same direction, meaning that many designers use design tools for the three purposes while few use design tools for only one purpose. Interrelated usage pattern is a key consideration for UI. In addition, design tool use for the three purposes increases as experience increases and for designers affiliated with organizations with 51~100 designers. This implies that the software-based design tool developers need to develop tools that can be easily used by large organizations (51~100 designers). Features that were analyzed to be important for prediction will be helpful in deriving insights for enhancing design tools: relationship between features and emphasis of particular feature per segment.

In terms of methodology, we abided by the basic process of data analysis, exploratory data analysis, and classification modeling. The study, therefore, can serve as an integrated case of data analysis. We inspected histograms, box plots, and correlation analysis for exploratory data analysis, while using random forest model for accurate prediction and logistic analytics for statistical significance of different variables. Deriving important features using random forest and identifying statistically significant features using logistic analytics can serve as a reference for similar research.

Despite the implications and methodological contributions, there are few limitations to this study. First, the data analyzed is not a randomly sampled data. While the survey was conducted for designers participating in a design community, many designers did not respond and hence the data resembles voluntary sampled data. Some researchers point out that voluntary samples tend to be lenient on investigations targeting the respondents themselves while being critical of investigations on products[63,64]. To overcome these limitations, this study filtered incomplete responses and responses of designers from unfit roles/organizations. Nevertheless, the data is limited in that it is not a randomly sampled data. The high proportion of PD and UX role in the sample is a limitation of this survey.

If the design tool is to be one of the products, the results of this study can be evaluated as a relatively conservative reflection of the positive aspects of the design tool.

Second, it is also possible to argue that the designers can be more creative and collaborative even if the design tools are not used for the three purposes investigated in this study. This study does not measure if this argument is valid or not. Furthermore, this study does not analyze if there is a statistically significant difference between deliverables of designers that use design tools for the three purposes and those of designers that do not. This is a topic for further study.

Third, collaboration can occur between designers, between designers and developers, and between designers and users. This study focused on the two and excluded the third [65]. If the third type of collaboration can be included, it can be included as designer activity to complement creativity. This observation will be proposed to UX Tools survey hosts and hopefully will be reflected in the future surveys.

The themes that were not fully investigated in this research are topics for further research. Overall, we observed that many designers use software-based design tools for the three purposes and hypothesized that the software-based design tools reinforce or promote these activities. This merits further research. Furthermore, an individual designer’s ability to improve overall with advances in design tools needs requires further investigation to determine whether the advances in design tools positively or negatively impact the creativity of an individual designer [1]. In the future, it may be possible to supplement the limitations of this study by using experimental or qualitative research.

As the knowledge industry and platform businesses advance, traditional products and services are commoditized. Meanwhile, as product life cycles matures, inter-firm competitions are intensified, and this calls for differentiation to survive in the market. In this context, designers are important entities for differentiation and innovation. Design organizations need to reexamine their evolution path (i.e., how they should it change from the past to present) and their differentiation strategies for design. This research analyzed design tool usage patterns and identified that design strategies and management need to be tailored to fields of design and the characteristics of the organization. If the organization’s objective is to foster the capability of individual designers and by this achieve the firm’s objectives, the organization needs to consider how designer activities, design tools, and firm objectives are interrelated within the framework of innovation. This study explored ideation as a creativity research theme and collaboration as an execution research theme within the innovation framework. Developing new design tools changes the course of design organization strategy and management. Therefore, the firms need to delve into how such development ultimately impacts their objectives.

Acknowledgments

: This research was supported by Kumoh National Institute of Technology (2019-104-049).

Conflicts of Interest

: The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. P. Schenk, "Inspiration and ideation: Drawing in a digital age," Design Issues, vol. 30, no. 2, pp. 42-55, 2014, doi: https://doi.org/10.1162/DESI_a_00261.
  2. D. McLean, "Adobe Photoshop and Illustrator techniques," Journal of Audiovisual Media in Medicine, vol. 25, no. 2, pp. 79-81, 2002, doi:https://doi.org/10.1080/01405110220140865.
  3. H. A. Jouybari and A. Farahnaky, "Evaluation of Photoshop software potential for food colorimetry," Journal of Food Engineering, vol. 106, no. 2, pp. 170-175, 2011, doi: https://doi.org/10.1016/j.jfoodeng.2011.02.034.
  4. C. Chen and F. Min. "Pro/E software applications in industrial product design," in 2011 International Conference on System science, Engineering design and Manufacturing informatization. IEEE, 2011, doi:10.1109/ICSSEM.2011.6081171
  5. R. E. Wendrich, F. V. Houten, H. Tragter, and K. Keler, "Bridging the Design Gap: Towards an Intuitive Design Tool," Proc. of the ICSID Wrld. Dgn. Con, 2009.
  6. Z. Howard and G. Melles. Beyond designing: roles of the designer in complex design projects. in Proceedings of the 23rd Australian Computer-Human Interaction Conference, 2011, doi: https://doi.org/10.1145/2071536.2071560.
  7. A. Somech and A. D. Zahavy, "Translating team creativity to innovation implementation: The role of team composition and climate for innovation," Journal of management, vol. 39, no. 3, pp. 684-708, 2013, doi: https://doi.org/10.1177/0149206310394187.
  8. M. A. West, "Sparkling fountains or stagnant ponds: An integrative model of creativity and innovation implementation in work groups," Applied psychology, vol. 51, no. 3, pp. 355-387, 2002, doi: https://doi.org/10.1111/1464-0597.00951.
  9. J. Guo, Q. Su, and Q. Zhang, "Individual creativity during the ideation phase of product innovation: An interactional perspective," Creativity and Innovation Management, vol. 26, no. 1, pp. 31-48, 2017, doi: https://doi.org/10.1111/caim.12205.
  10. D. Mascia, M. Magnusson, and J. Bjork, The role of social networks in organizing ideation, creativity and innovation: An introduction, 2015, doi: https://doi.org/10.1111/caim.12111.
  11. B. Weber and S. Heidenreich, "When and with whom to cooperate? Investigating effects of cooperation stage and type on innovation capabilities and success," Long Range Planning, vol. 51, no. 2, pp. 334-350, 2018, doi: https://doi.org/10.1016/j.lrp.2017.07.003.
  12. C. A. Sacramento, M. W. S. Chang, and M.A. West, Team innovation through collaboration, in Innovation through collaboration, Emerald Group Publishing Limited, pp. 81-112, 2006, doi: https://doi.org/10.1016/S1572-0977(06)12004-X
  13. O. Akin, "Case-based instruction strategies in architecture," Design Studies, vol. 23, no. 4, pp. 407-431, 2002, doi:https://doi.org/10.1016/S0142-694X(01)00046-1.
  14. J. Li, G. W. Tigwell, and K. Shinohara. "Accessibility of High-Fidelity Prototyping Tools," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021, doi: https://doi.org/10.1145/3411764.3445520.
  15. L. Y. Hwa, " A Study of the Visual Elements of Image Editing Design-Focused on tvN Entertainment Programs," Formative Media Studies, vol. 20, no. 2, pp. 138-146, 2017, UCI(KEPA): I410-ECN-0102-2018-600-000610307
  16. Y. Lee, "An Analysis of Expression According to Elements of Motion Graphic," Journal of Korea Design Knowledge, vol. 22, pp. 143, 2012, doi: 10.17246/jkdk.2012..22.014
  17. Y. Cho, "A Study on the Classification System of the Motion in Motion Graphics,". Journal of Digital Design, vol. 9, no. 1, pp. 119-130, 2009, doi: https://doi.org/10.17280/jdd.2009.9.1.013.
  18. R. J. Lawrence, "Architectural design tools: simulation, communication and negotiation," Design Studies, vol. 14, no. 3, pp. 299-313, 1993, doi: https://doi.org/10.1016/0142-694X(93)80026-9.
  19. R. Ibrahim and F. P. Rahimian, "Comparison of CAD and manual sketching tools for teaching architectural design," Automation in Construction, vol. 19, no. 8, pp. 978-987, 2010, doi: https://doi.org/10.1016/j.autcon.2010.09.003.
  20. S. Lee and J. Yan, "The impact of 3D CAD interfaces on user ideation: A comparative analysis using SketchUp and Silhouette Modeler," Design Studies, vol. 44: pp. 52-73, 2016, doi: https://doi.org/10.1016/j.destud.2016.02.001.
  21. B. Jonson, "Design ideation: The conceptual sketch in the digital age," Design Studies, vol. 26, no. 6, pp. 613-624, 2005, https://doi.org/10.1016/j.destud.2005.03.001.
  22. M. C. Yang, "Observations on concept generation and sketching in engineering design," Research in Engineering Design, vol. 20, no. 1, pp. 1-11, 2009, doi: https://doi.org/10.1007/s00163-008-0055-0.
  23. A. Ivanov and D. Cyr, "The concept plot: a concept mapping visualization tool for asynchronous web-based brainstorming sessions," Information Visualization, vol. 5, no. 3, pp. 185-191, 2006, doi: https://doi.org/10.1057/palgrave.ivs.9500130.
  24. C. Stones and T. Cassidy, "Seeing and discovering: How do student designers reinterpret sketches and digital marks during graphic design ideation?" Design Studies, vol. 31, no. 5, pp. 439-460, 2010, doi: https://doi.org/10.1016/j.destud.2010.05.003.
  25. C. K. Allison and N. A. Stanton, "Ideation using the "Design with Intent" toolkit: A case study applying a design toolkit to support creativity in developing vehicle interfaces for fuel-efficient driving," Applied ergonomics, vol. 84, pp. 103026, 2020, doi: https://doi.org/10.1016/j.apergo.2019.103026.
  26. M. Hanzl, "Information technology as a tool for public participation in urban planning: a review of experiments and potentials," Design Studies, vol. 28, no. 3, pp. 289-307, 2007, doi: https://doi.org/10.1016/j.destud.2007.02.003.
  27. J. P. McCormack, J. Cagan, and C. M. Vogel, "Speaking the Buick language: Capturing, understanding, and exploring brand identity with shape grammars," Design Studies, vol. 25, no. 1, pp. 1-29, 2004, doi: https://doi.org/10.1016/S0142-694X(03)00023-1.
  28. L. Heiss and O. Kokshagina, "Tactile co-design tools for complex interdisciplinary problem exploration in healthcare settings," Design Studies, vol. 75, 2021, doi: https://doi.org/10.1016/j.destud.2021.101030.
  29. G. Dove, S. J. Abildgaard, M. M. Biskjaer, N. B. Hansenm, B. T. Chrisrensen, and K. Halskov, "Grouping notes through nodes: The functions of Post-It notes in design team cognition," Design Studies, vol. 57, pp. 112-134, 2018, doi: https://doi.org/10.1016/j.destud.2018.03.008.
  30. K. Shroyer, T. Lovins, J. Turns, M. E. Cardella, and C. J. Atman, "Timescales and ideaspace: An examination of idea generation in design practice," Design Studies, vol. 57: pp. 9-36, 2018, doi: https://doi.org/10.1016/j.destud.2018.03.004.
  31. T. Dorta, G. Kinayoglu, and S. Boudhraa, "A new representational ecosystem for design teaching in the studio," Design Studies, vol. 47, pp. 164-186, 2016, doi: https://doi.org/10.1016/j.destud.2016.09.003.
  32. D. Y. Jung and S. I. Kim, "The development direction and Analysis on Graphic authoring tool according to Design work frame-Focused on Interface design," Journal of the Korea Convergence Society, vol. 8, no.11, pp. 233-240, 2017, doi: https://doi.org/10.1016/j.destud.2010.09.006.
  33. G. Goldschmidt and A. L. Sever, "Inspiring design ideas with texts," Design Studies, vol. 32, no. 2, pp. 139-155, 2011, doi: https://doi.org/10.1016/j.destud.2010.09.006.
  34. T. Kurtoglu, M. I. Campbell, and J. S. Linsey, "An experimental study on the effects of a computational design tool on concept generation," Design Studies, vol. 30, no. 6, pp. 676-703, 2009, doi: https://doi.org/10.1016/j.destud.2009.06.005.
  35. J. Han, H. Forbes, and D. Schaefer, "An exploration of how creativity, functionality, and aesthetics are related in design," Research in Engineering Design, pp. 1-19, 2021, doi: https://doi.org/10.1007/s00163-021-00366-9
  36. C. M. Snider, S. J. Culley, and E. A. Dekoninck, "Analysing creative behaviour in the later stage design process," Design Studies, vol. 34, no. 5, pp. 543-574, 2013, doi: https://doi.org/10.1016/j.destud.2013.03.001.
  37. M. N. Folkmann, "Enabling creativity. Imagination in design processes," in DS 66-2: Proceedings of the 1st International Conference on Design Creativity, ICDC 2010. 2010.
  38. D. H. Kim, "Solutions for Design Creativity Barriers," JOURNAL OF THE KOREA CONTENTS ASSOCIATION, vol. 20, no. 11, pp. 15-27, 2020, doi: https://doi.org/10.5392/JKCA.2020.20.11.015
  39. K. Dorst and N. Cross, "Creativity in the design process: Co-evolution of problem-solution," Design Studies, vol. 22, no. 5, pp. 425-437, 2001, doi: https://doi.org/10.1016/S0142-694X(01)00009-6.
  40. A. H. Musta'amal, E. Norman, M. K. Jabor, and Y. Buntat, "Does CAD really encourage creative behaviors among its users: a case study," Procedia-social and behavioral sciences, vol. 56, pp. 602-608, 2012, doi: https://doi.org/10.1016/j.sbspro.2012.09.694
  41. C. Silvestri, R. Motro, B. Maurin, and B. D. langley, "Visual spatial learning of complex object morphologies through the interaction with virtual and real-world data," Design Studies, vol. 31, no. 4, pp. 363-381, 2010, doi: https://doi.org/10.1016/j.destud.2010.03.001.
  42. B. Lawson, "CAD and creativity: does the computer really help?," Leonardo, vol. 35, no. 3, pp. 327-331, 2002, doi: https://doi.org/10.1162/002409402760105361.
  43. P. F. Culverhouse, "Constraining designers and their CAD tools," Design Studies, vol. 16, no. 1, pp. 81-101, 1995, doi: https://doi.org/10.1016/0142-694X(95)90648-Y.
  44. K. P. Schulz, S. Geithner, C. Woelfel, and J. Krzywinski, "Toolkit-based modelling and serious play as means to foster creativity in innovation processes," Creativity and innovation management, vol. 24, no. 2, pp. 323-340, 2015, doi: https://doi.org/10.1111/caim.12113
  45. S. Bodker, C. Nielsen, and M.G. Petersen. "Creativity, cooperation and interactive design," in Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques. 2000, doi: https://doi.org/10.1145/347642.347757.
  46. M. Steen, M. Manschot, and N. De Koning, "Benefits of co-design in service design projects," International Journal of Design, vol. 5, no. 2, 2011.
  47. M. Steen, "Co-design as a process of joint inquiry and imagination," Design Issues, vol. 29, no. 2, pp. 16-28, 2013, doi: https://doi.org/10.1162/DESI_a_00207.
  48. Y. Lee, " Design participation tactics: the challenges and new roles for designers in the co-design process," Codesign, vol. 4, no. 1, pp. 31-50, 2008, doi: https://doi.org/10.1080/15710880701875613.
  49. M. Kleinsmann and R. Valkenburg, "Barriers and enablers for creating shared understanding in co-design projects," Design studies, vol. 29, no. 4, pp. 369-386, 2008, doi: https://doi.org/10.1016/j.destud.2008.03.003.
  50. M. Twidale, T. Rodden, and I. Sommerville, "The Designers' Notepad: Supporting and understanding cooperative design," in Proceedings of the Third European Conference on Computer-Supported Cooperative Work 13-17 September 1993, Milan, Italy ECSCW'93. 1993. Springer, doi: https://doi.org/10.1007/978-94-011-2094-4_7.
  51. A. Jones, V. Thoma, and G. Newell, "Collaboration constraints for designers and developers in an agile environment," in Proceedings of the 30th International BCS Human Computer Interaction Conference 30. 2016, doi: https://doi.org/10.14236/ewic/HCI2016.37.
  52. N. Maudet, G. Leiva, M. B. Lafon, and W. Mackay "Design Breakdowns: Designer-Developer Gaps in Representing and Interpreting Interactive Systems," in Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, 2017, doi: https://doi.org/10.1145/2998181.2998190.
  53. G. Nudelman, Lean UX communication strategies for success in large organizations. Interactions, vol. 25, no. 5, pp. 80-82, 2018, doi: https://doi.org/10.1145/3236683.
  54. S. Morson, "Sketch Resources, in Designing for iOS with Sketch," Springer. pp. 167-173, 2015, doi: https://doi.org/10.1007/978-1-4842-1458-9_11.
  55. J. T. Behrens, "Principles and procedures of exploratory data analysis," Psychological Methods, vol. 2, no. 2, pp. 131, 1997, doi: https://doi.org/10.1037/1082-989X.2.2.131.
  56. C. Chatfield, "Exploratory data analysis," European journal of operational research, vol. 23, no. 1, pp. 5-13, 1986, doi: https://doi.org/10.1016/0377-2217(86)90209-2.
  57. J. W. Tukey, "Analyzing data: Sanctification or detective work?" American Psychologist, vol. 24, no. 2, pp. 83, 1969, doi: https://doi.org/10.1037/h0027108.
  58. D. R. Cutler, T. C. Edwards, K. H. Beard, and A. Cutler, "Random forests for classification in ecology," Ecology, vol. 88, no. 11, pp. 2783-2792, 2007, doi: https://doi.org/10.1890/07-0539.1.
  59. L. Breiman, " Random forests," Machine Learning, vol. 45, no. 1, pp. 5-32, 2001, doi: https://doi.org/10.1023/A:1010933404324.
  60. D. G. Kleinbaum, Logistic regression, Springer, 2002.
  61. J. A. Swets, "Measuring the accuracy of diagnostic systems. Science," vol. 240, no. 4857, pp. 1285-1293, 1988, doi: https://doi.org/10.1126/science.3287615.
  62. J. N. Mandrekar, "Receiver operating characteristic curve in diagnostic test assessment," Journal of Thoracic Oncology, vol. 5, no. 9, pp. 1315-1316. 2010, doi: https://doi.org/10.1097/JTO.0b013e3181ec173d
  63. T. Mallett, "Retaining virtue in a virtual world: e-surveys in public policy research," in Professional Market Research Society Annual Conference, CFIB. 2001.
  64. G. W. Yun and C.W. Trumbo, "Comparative response to a survey executed by post, e-mail, & web form," Journal of computer-mediated communication, vol. 6, no. 1, pp. JCMC613, 2000, doi: https://doi.org/10.1111/j.1083-6101.2000.tb00112.x.
  65. C. Bassetti, M. Sciannamblo, P. Lyle, M. teli, S. D. Paoli, and A. D. Angeli, "Co-designing for common values: creating hybrid spaces to nurture autonomous cooperation," Co Design, vol. 15, no. 3, pp. 256-271, 2019, doi: https://doi.org/10.1080/15710882.2019.1637897.