Results in Engineering 8 (2020) 100175 Contents lists available at ScienceDirect Results in Engineering journal homepage: www.editorialmanager.com/rineng/Default.aspx Fostering product innovations in software startups through freelancer supported requirement engineering Varun Gupta a,b,*, Jose Maria Fernandez-Crehuet a, Thomas Hanne b, Rainer Telesko b a Departamento de Ingeniería de Organizaci�on, Administraci�on de empresas y Estadística, Universidad Polit�ecnica de Madrid, Spain b School of Business, University of Applied Sciences and Arts Northwestern Switzerland, Switzerland A R T I C L E I N F O Keywords: Software engineering Requirement engineering Software startups Freelancer Open innovation Co-creation Value proposition innovation * Corresponding author. Departamento de Ingeni E-mail addresses: varun.gupta@alumnos.upm.es rainer.telesko@fhnw.ch (R. Telesko). https://doi.org/10.1016/j.rineng.2020.100175 Received 24 August 2020; Received in revised form 2590-1230/© 2020 The Author(s). Published by Els nc-nd/4.0/). A B S T R A C T This research paper explores the involvement of freelancers in requirement engineering activities to continuously innovating the value propositions and utilizing their expertise in various requirement engineering tasks. This paper reports the case study conducted with the startups that involve freelancers for the requirement engineering activities. The findings are then compared with the literature to explore the freelancer supported requirement engineering domain. Results indicate that the freelancers could help innovate value proposition by providing different perspectives of the global segments and also expertise in executing requirement engineering activities. The freelancers have varying levels of involvement in requirement engineering activities depending on on startup contexts and is highly challenged by various inhibitors. The inhibitors include difficulty to select freelancers optimally, ensuring their long term association for continuous rework arising because of continuous learnings in the market, building trust, mechanism to integrate their perspective, establishing communication, negotiations and strategic pricings. However, there is a need to optimally establish the freelancer involvement from beginning of the startup life cycle with a promise for long term benefits in exchange for their trustworthy and accurate perspectives, which is harder to get by involving crowds of customers due to resource limitations. Further research is required to investigate how freelancers could represent the samples of globally distributed customer segments as input source of information on one side and on another side become startup team representatives to establish direct interactions with global customer segments. 1. Introduction The requirement engineering is one of the most crucial stages of the software engineering process that involves the identification of the needs (pain points and benefits) of the customers, which are further refined, validated with the possible customers, prioritized, documented, imple- mented & released to the market. The requirement engineering process model is repeatedly executed to foster software evolution continuously as per customer feedback. The value proposition specifies what value the software is delivering to its customers in terms of its ability to solve their problems (i.e. fulfill their needs) in an efficient manner. The features of the software must address the pain points of the customers and create gains (benefits) to them. The customer needs are continuously changing and due to competitor pressure, the companies must maintain competi- tive advantage by continuously innovating the product value proposi- tion. In other words, the software companies will have to continuously ería de Organizaci�on, Administra (V. Gupta), josemaria.fernandez-c 26 September 2020; Accepted 2 evier B.V. This is an open access a identify the creative ideas about what new features to be provided or how existing ones could be optimized to improve performance, to enhance value to the customer (product innovation). Value proposition innova- tion is all about continuously providing value to the customer by meeting all his changing needs in a better way. Product innovations means inventing creative ideas resulting in new products or change in existing products that creates value to its customers i.e. has an ability to satisfy the customers. Identifying the new features of the existing product to enhance value to the customer or inventing the ways of improving its performance, addresses needs of the customer and hence leads to value proposition innovation. However, there are many other ways of enhancing the customer value which may include lowering development costs (which finally lowers price) and improving benefits (for instance emotional benefits, benefits by improving distribution channels, reducing customer hassles in using the product etc.), which strongly af- fects the value proposition. Interactions with the customer and their ci�on de empresas y Estadística, Universidad Polit�ecnica de Madrid, Spain: rehuet@upm.es (J.M. Fernandez-Crehuet), thomas.hanne@fhnw.ch (T. Hanne), 9 September 2020 rticle under the CC BY-NC-ND license (http://creativecommons.org/licenses/by- mailto:varun.gupta@alumnos.upm.es mailto:josemaria.fernandez-crehuet@upm.es mailto:thomas.hanne@fhnw.ch mailto:rainer.telesko@fhnw.ch http://crossmark.crossref.org/dialog/?doi=10.1016/j.rineng.2020.100175&domain=pdf www.sciencedirect.com/science/journal/25901230 www.editorialmanager.com/rineng/Default.aspx https://doi.org/10.1016/j.rineng.2020.100175 http://creativecommons.org/licenses/by-nc-nd/4.0/ http://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.1016/j.rineng.2020.100175 V. Gupta et al. Results in Engineering 8 (2020) 100175 continuous feedback is the valuable source for innovating the value proposition. Both evolutionary Requirement engineering and Value proposition innovation tries to uncover the needs of the customer and release the product that addresses their problems. The evolutionary requirement engineering activity is thus another term to be used inter- changeably with the value proposition innovation. Requirement engineering is a highly customer centric and co-creation activity that involves high collaboration with the customers (and users). The customers could be globally distributed and hence outside the reach of the startups, which limits their continuous involvement in requirement engineering because of accessibility issues. Startups must continuously involve customers in requirement engineering activity because every step of interaction with the customer is a way of testing and building the assumptions (also called a hypothesis) about problems, solutions or any other aspects related to business models. The strong interaction with the customers leads to the better understanding of the market and the gen- eration of the creative innovative ideas leading to the incremental, component, architectural or radical innovations. Limited access to the global segments may hinder the generation of creative innovative ideas because of low understanding of the underserved segment problems. Further, limited resources inhibit the ability of the startups to undertake all requirement engineering activities inhouse. The activities are con- ducted with limited understanding of the problem domain (due to less interaction with global segments). The process of continuously evolving the software product happens throughout the software life cycle due to ever changing customer needs and changing business environment. Thus, the continuous support of customers to continuously innovate value proposition is very much required. Freelancers are the self-employed persons, providing the solutions to the software engineering tasks as requested by the software company (which lacks the ability to undertake it in house), by using their niche skills and expertise. The focus of this paper is to identify that how the startups could foster innovations by involving freelancers during requirement engineering activity for fostering innovation i.e. changes in value proposition that generates value to the customer and involving them in different requirement engineering activities where startups lacks inhouse expertise. Former is achieved as the freelancers could help startups to provide the perspectives about the needs of such global seg- ments, driven by their previous domain expertise, their understanding of unique needs of the segments local to their geographical regions and understandings brought by the face to face interaction with segments as startup team member, could help startup teams take diverse perspectives in consideration to innovate the value proposition. Latter is achieved by outsourcing the requirement engineering tasks to the freelancers thereby bridging the skill lacking inhouse. This paper highlights the ongoing research in freelancer supported Requirement Engineering to investigate the various requirement engi- neering activities where freelancers could be involved at varying levels of associations. The research question for which this study tries to find answer could be formulated as RQ: What is state of affairs in the freelancer supported requirement engineering in the context of the software startups as revealed by literature and in real context? This paper meets its research objective by undertaking the following steps: � Comparing the three empirical studies (Study 1, 2 and 3) that report the aspects related to the state of art of requirement engineering and freelancer’s involvement in the context of software startups, as available in the literature. The result is the identification of the in- hibitors and catalysts for employing the freelancers in requirement engineering. � Another case study (Study 4) is conducted with the startups that had involved (or currently involving) the freelancers for Requirement Engineering activity. The lead to elaboration of the findings as revealed by the literature previously. 2 This paper is structured as follows; the related work related to the three empirical studies are presented in section 2; comparative analysis of the empirical studies are presented in section 3; the conducted case study is presented in Section 4; the findings are discussed in section 5; future research work planned is given in Section 6 and finally paper is concluded in Section 7. 2. Related work 2.1. Freelancers in software development: systematic mapping study (study 1) Gupta et al. [1] conducted a systematic mapping study to provide state of art in freelancer supported software development and further to identify their involvement in the software startups context. This research was based on the systematic mapping guidelines as given in Ref. [2]. The results indicated that the studies reported the use of freelancers for generic software development (78%) rather for individual development activities. The reported challenges include Collaboration & Coordination (33%), Developer Recommendation (or selection) (19%), Team Formu- lation (14%), Task Recommendation (allocation) (14%), Task Decom- position (11%), Privacy & Security (Confidentiality)(11%), Budget Estimation (8%), Recognition (8%), Trust Issues (8%), Market Dynamism (6%), Intellectual Property Issues (6%), Participation of Crowd Worker (6%) and Capacity Utilization (3%). These challenges are highly inter- active, and each challenge impacts all other challenges. Most of the studies are validation approaches (72%) followed by empirical studies (25%) and the solutions approaches based on toy examples (3%). Recent focus of the researchers (total 7 studies in 2019) is on generic software development handling the collaboration & coordination (3 studies out of 7), Developer recommendation (2 studies out of 7) and task recom- mendation (2 studies out of 7). 2.2. Freelancing models of software startups: empirical study (study 2) Gupta et al. [3] conducted the exploratory case study to analyze the startup strategies to involve freelancers in various software development tasks along with the associated challenges and business impacts. The research study was based on the case study guidelines proposed in Runeson and H€ost [4] to conduct the exploratory study. The results indicated that the freelancer association strategy is either Task based (ending with completion of task to be outsourced), Panel based (outsourcing task to the panel of freelancers associated with startup without any employment benefits) or hybrid. The startups want to harness the power of crowdsourcing for freelancer selections but the choice is limited due to lack of resources. Uncertainties, terminology issues, high technical debt, lack of documentation, lack of systematic decision making processes, lack of resources and lack of brand values are the main inhibitors for associations with freelancers. The availability of good freelancers and long term associations positively impacts startups. 2.3. Requirement engineering in software startups: systematic mapping study (study 3) Gupta et al. [5] conducted the systematic mapping study to present the state of art of the requirement engineering research in the context of startups- Systematic mapping study was conducted by following the guidelines as given in Ref. [2] and the updated guidelines as dissemi- nated in Petersen et al. [6]. The results indicated the researchers focused on the generic requirement engineering process (40%) (rather on indi- vidual activities) followed by product validation (20%), requirement elicitation (17.5%) and requirement validation (12.5%). Requirement prioritization and documentation are the least focused activities (5% each). Major studies are empirical research studies (85%) followed by validation approaches (solutions with laboratory validations) (10%) and experience (5%). The solutions, validated solutions and solutions Table 1 Comparative analysis. Study Key Results (Requirement Engg. Related only) Inhibitors for freelancers based RE Catalysts for freelancers based RE Study 1 Main Focus Research focused on generic software engineering rather individual activities. Limited studies focused on freelancing in Requirement Engineering. None. Challenges Collaboration & Coordination (33%), Developer Recommendation (19%), Team Formulation (14%), Task Recommendation (14%), Task Decomposition (11%), Privacy & Security (11%), Budget Estimation (8%), Recognition (8%), Trust Issues (8%), Market Dynamism (6%), Intellectual Property Issues (6%), Participation of Crowd Worker (6%) and Capacity Utilization (3%). These challenges hinder the selection of good freelancer and maintaining long term relationships that is necessary in requirement engineering. None. Study 2 Association Models Task based models (ending with completion of task to be outsourced), Panel based models (outsourcing task to the panel of freelancers associated with startup without any employment benefits) or hybrid models. Lack of reputation hinders the formulations of panels. Limited resources hinder the idea of the crowdsourcing based freelancing. During growing phase, customized crowdsourcing could be feasible. Individual selection of freelancers (especially college interns and from known networks) could be employed. Challenges for association Uncertainties, terminology issues, high technical debt, lack of documentation, lack of systematic decision makings processes, lack of resources and lack of brand values. Challenges limit the use of freelancers for Requirement Engineering. None. Outcome The availability of good freelancers and long term associations positively impacts startups. None. Motivation to include freelancers in Requirement Engineering. Study 3 Main Focus Focused on generic requirement engineering process (40%). The studies do not report any freelancer involvement in requirement engineering activities of the software startups Limited studies neither provides empirical guidelines not the validated solutions that could be helpful to startups. None. Research Focus Major focus of studies is to attain a good product/market fit (62.5%) followed by process improvement (27.5%) and least focus on processes (10%). None. Studies could be worked for extension in requirement engineering contexts using freelancers. V. Gupta et al. Results in Engineering 8 (2020) 100175 validated in real settings are very limited. Major focus of studies is to attain a good product/market fit (62.5%) followed by process improve- ment (27.5%) and least focus on studying ongoing processes (10%). The studies analyzed in the systematic mapping study do not report any freelancer involvement in requirement engineering activities of the software startups. Further the research focus in this area in form of validated solutions, is not enough to support startups to overcome the failure related issues related to requirement engineering processes. 3. Freelancers in requirement engineering: comparative analysis of empirical studies The three empirical studies i.e. study 1, study 2 and study 3 (section 2) are subjected to comparative analysis from the viewpoint of identifi- cation of the inhibitors and catalysts for the freelancer involvement in requirement engineering and is presented in Table 1. 4. Case study: involving freelancers in requirement engineering The case study has the objective to identify the software practices and lessons learned by the studies startups to involve freelancers for various requirement engineering activities. To meet the stated objectives, the researchers performed purposive sampling technique to sample the startup cases based on their ability to fulfil the purpose of the research study. The case study follows the guidelines as proposed in Refs. [4]. The difficulty in finding the startups that involve freelancers during requirement engineering is because majority of the software startups usually perform the requirement engineering activities (at least till pro- duct/market fit is attained) by an in house team comprising founders and employees. Employing freelancers during different requirements engi- neering activities has been very unusual for the majority of startups in the startup community because they dont want to outsource their core business operation. This activity brings to the startup team a lot of learning about the business model assumptions (including the value propositions) as a result of direct interactions with the customers and it also helps them to establish relationships with the potential customers leading into future sales. This case study tries to find answers to the following research questions: RQ. 1 How do the startups involve freelancers during requirement engineering activities? RQ. 2 What challenges and benefits do the startups observed due to associations with the freelancers? RQ. 3 What are the challenges to continuously innovate the software product unlike the established companies? The interviews and observations were conducted to gather data from the startups, based in India and Switzerland. Founder of both startups and Product Manager (experienced developer designated as product manager) were interviewed. The field notes and interview sessions were transcripted at the time of interview only and subjected to grounded theory to generate meaningful responses to formulated research ques- tions. Interview sessions were held twice, with swiss based startups during March 2020 face to face and during April through skype. The interview session was held with Indian startup through Online video conferencing tools during March and April 2020. The reason for involving these startups was that they have involved the freelancers during the requirement engineering activity. The results of the interview sessions are briefly described below: Supplementary video related to this article can be found at https ://doi.org/10.1016/j.rineng.2020.100175. 4.1. RQ. 1 how do the startups involve freelancers during requirement engineering activities? The Swiss startup involves freelancers to get better insights about the customer segments served by the startup. This startup had one customer 3 https://doi.org/10.1016/j.rineng.2020.100175 https://doi.org/10.1016/j.rineng.2020.100175 V. Gupta et al. Results in Engineering 8 (2020) 100175 segment in the UK and hence the founder decided to employ Two free- lancers (with domain expertise in medical projects and based in the UK) to validate the learnings the founder got with interaction with the cus- tomers. In other words, the freelancer involvement during the require- ment gathering phase was an exercise to validate the learning model rather completely dependent on freelancer inputs. The founder stated that “The involvement of freelancers during understanding customer needs (exploring problem domain) is very important as it helps to take the perspective of freelancers into consideration, that draws upon their previous project ex- periences and understanding of the market needs local to them (but global to the startups)”. Also, the freelancer perspectives could avoid the mistakes in experimenting with the wrong customer segments located in different global segments. As per the product manager of the swiss startup, “Freelancers perspectives saved a lot of our resources that would otherwise have been wasted on experimenting with the wrong market segments. We just need to invest small efforts to validate the freelancer perspectives that seem to be negative and conflicting in the first instance to the startups”. The free- lancer’s inputs were also used to validate requirements, prioritize re- quirements and product validation. The swiss startup does not involve freelancers for the Minimal Viable Product (MVP) development because of the need to frequently evolve the product as per continuous customer feedback. Indian startups involved the college students as freelancers for various Requirement engineering activities like requirement engineering (or exploring problem domain), validating requirements (by using pro- totypes), identify high priority features (requirement prioritization), creating MVP and MVP evolution. As per the founder of the Indian startup, “The involvement of freelancers especially college graduates help highly resource constrained startups to quickly reach markets and gain suffi- cient flexibility to quickly reach to failures”. The main reasons for involving the college student were the factors related to outsourcing costs, long term association promise from college students, flexibility to execute the same task again to incorporate new learnings etc. 4.2. RQ. 2 what challenges and benefits do the startups observed due to associations with the freelancers? The startups reported that involvement of the freelancers helps to involve their expertise and perspectives to quickly validate your problem domain understandings before release to the market. Another benefit is freelancers help to spread information about the startup to their networks very quickly through Word of mouth. Freelancers if selected properly could help to reduce failures chances and increase successful releases leading to saving development costs. Freelancer associations also helps startup teams to avoid costs of multiple travels for customer sites for observations and difficulty in getting all perspectives from customers. Freelancers have insights because of familiarity with local customer segment unique needs, previous project domain expertise and perhaps their ability to have the face to face interactions with the customers as startup teammembers. However, the biggest challenge is to identify high quality freelancer at reasonable rates, ensuring their long term associa- tion for continuous rework arising because of continuous learnings, trusting the freelancers, merging their opinions with the judgements of the startup teams (collected through direct interactions with the cus- tomers), establishing communication with them, difficulty in negotia- tions due to startup reputation issues, pricing issues (also from strategic point of view) etc. The continued involvement of freelancers is required as the product evolves continuously which requires the involvement of freelancers. However, many such challenges are resolved with the maturity of the startups i.e. startups become familiar with problem domain and markets, making it easier to incorporate freelancers’ inputs. As per swiss startup founder “The biggest challenge is there are no available guidelines and no solutions, and everything is your own judgement and risk”. 4 4.3. RQ. 3 what are the challenges to continuously innovate the software product unlike the established companies? As per the swiss startup founder “The biggest challenge for startups to incrementally innovate the product is to collect and analyze the diverse ideas”. In matured companies, the plenty of resources allows them to establish the channels to collect and analyze the diverse ideas collected from the crowd of their customers (or potential customers) which is the biggest challenge for the startups. Startups face issues like difficulty motivating customers to engage in the ideation process, establishing multiple feed- back channels that are subjected to automatic analysis, validating the ideas and much more. As per the Indian Startup founder “The biggest challenge is that you have to balance existing customers and potential cus- tomers, as your resources are limited, and the future is uncertain”. The interviewed startups also stated that crowdsourcing seems to be a biggest challenge for them due to lack of experience and resources. For the majority of startups, the freelancer’s involvement in requirement engineering is a bit unusual because during initial stages founders like to personally interact with customers to better understand their needs and establish personal relations that help them turn into their customers. As per Indian startup founder “The use of freelancers in requirement engineering activity is very unusual as the startups identify the software requirements through build-measure-learn loop (experimentations) by continuous in- teractions with the customers through different channels like online meetings, face to face meetings, observations, personal experiences etc. Also the limited resources and high uncertainties does not allow the startups to amplify the risks by undertaking requirement engineering (especially elicitation, prioritization and validation) using external crowdsourcing (using freelancers) as it is very costly to invest in crowdsourcing platforms, less intrinsic motivation among freelancers to participate (due to brand issues and less stakes in new product) and difficulty of startups to extrinsically motivate them with monetary benefits”. 5. Overall discussion This section answers the research question formulated in Section1, which is mentioned here as, RQ: What is state of affairs in the freelancer supported requirement engineering in the context of the software startups as revealed by literature and in real context? The analysis of the responses from two startups (Empirical study 4) suggest that freelancer’s involvement right from the beginning of the startup is beneficial for better exploring the problem domain from multiple perspectives, which may reduce the mistakes and better meet market needs. This is because requirement engineering needs long term associations with the freelancers to main- tain continuous evolution of the software in dynamic markets. However, individual selection of freelancers is the most suitable strategy compared to the crowdsourcing based freelancer involvement because lack of re- sources limits highly scalable crowdsourcing. Individual selection also depends on professional network and reputation of the startups. The various challenges associated with the freelancer involvement in soft- ware engineering tasks in startups as revealed by empirical study 1, 2 and 3 are also identified as the inhibiting factors in the studied startups (empirical study 4). Adding to this situation, lack of resources i.e. empirical guidelines, lessons learned, solution approaches etc. in the literature does not felicitates the decision makings in the software startups. The level of involvement of freelancers in requirement engi- neering either as expert or as virtual team member is a tradeoff situation, resolution of which is heavily dependent on the founder’s background, startups resources and long term vision. Involving the freelancers for Requirement Engineering means outsourcing the core business operation which increases the dependency of the startup teams on external collaboration. The studied startups had not completely outsourced the Requirement Engineering activity to the freelancers and their involve- ment is like an opinion provider to the startup team, helping them to provide fresh perspectives about the problem domain, providing opin- ions about requirement priorities etc. The partial involvement of the Table 2 Comparative study (explored). Comparative Study (Table 1) Case Study (Section 4) Agreement? Limited Empirical studies. No guidelines make it hard to apply best practices about which freelancer to apply and in which requirement engineering phase. Yes. Limited validation solutions. No available solutions make everything as new risky experience. Yes. Numerous Challenges like collaboration, coordination, selection etc. These challenges prevail in studies startups and it was hard for them to correctly identify freelancers. Yes. Limited crowdsourcing based freelancing Limited resources make it hard to apply crowdsourcing in early startup phases. Yes. Non crowdsourcing based freelancing This seems to be feasible solutions, but it depends on your existing network of relations. Yes. Freelancers could result in success. Freelancers helped by acting as customer sample and startup team member i.e. bridge between customer segment and startups. Yes. V. Gupta et al. Results in Engineering 8 (2020) 100175 freelancers helps the startups to get benefitted by freelancer perspectives and address the challenges with the outsourcing of the core business operation. In other words, startups do not lose anything with the accurate involvement of the freelancers. They enhance their success chances by involving freelancers and overcoming their resource limitations. There is an agreement between literature findings (conducted empirical studies) and the findings from the studied startups (Table 2). Table 2 compares the findings of comparative analysis of three empirical studies (Table 1) and case study conducted in section 4 to explore the information related to freelancer’s involvement in Require- ment Engineering. 6. Future directions of the research work The different activities of requirement engineering involve different challenges, so there is a need to conduct more research to identify the freelancer integration with individual activities overcoming the specific challenges. The research is required to identify that how startups could optimally select freelancer and motivate them to continuously contribute as startup team global member in evolving software value proposition. Further, what crowdsourcing lessons and support could be rented to the startups from the big matured companies, is an interesting research under investigation by the researchers. In future I plan to extend the empirical study 4 with more external startups as cases to check for generalizability. Also, I plan to include internal startups (negative cases) cases to study the variations in results as including them will diversifying the perspectives. This is because the internal startups have access to plenty of resources of parent company and the internal employees (and their existing networks) of parent company behave as freelancers (mostly driven by intrinsic motivations to contribute). The results will help to understand how the internal startups learned lessons could be mapped to external startups and how competitor collaboration could help both types of startups to create synergies by creating a mutual win-win situation. Another interesting work planned in future is to provide global startups (those targeting segments in multiple geographical locations) the ability to establish a virtual global team of employees, founders and freelancers that collectively performs requirement engineering and maps the diverse needs into a single collective set that maintains sustained innovations. However, the challenge is to identify individual freelancers (especially panel based freelancing method (refer to study 2)), motivate them to contribute at reasonable prices, streamlining the communica- tions (may be using user stories) and improving the decision makings by providing the set of rationale to help founder make particular decision. We are investigating the use of gamifications, storytelling and decision 5 support matrix to improve the decision makings, communications, mo- tivations and trustworthiness of freelancer information. Last we will investigate the effectiveness of the college graduates as freelancers for supporting the decision makings in the requirement engineering as we have evidences from the interviewed startups that if such graduates are chosen properly then they try to maintain long term relations (to support their future ongoing education or expectations for jobs with startups etc.) and they proves to be less expensive that highly reputable freelancers. 7. Conclusion The continuous innovation requires accurate collection of diverse ideas from stakeholders especially customers. Crowdsourcing for inno- vation seems to be limited in startup contexts except for customized flexible processes but accurate individual freelancer selections globally (as per segments locations) could be very fruitful to the startups. How- ever, the challenges pertaining to the startups makes it hard for them to select good freelancer and establish long term relationships, which is a necessary condition in evolving requirement engineering (as product need to be evolved continuously with dynamism in markets). Freelancers could help startups innovate their value propositions by playing a double role of customer segment representatives (providing ideas about unique needs of customers) and startup team representatives (establishing customer relations). There is a need optimally establish the freelancer involvement from beginning of the startups with a promise for long term benefits in exchange for their trustworthy and accurate perspectives that is required for open innovation, which is harder to get by involving crowds of customers due to resource limitations. There is a need for more empirical research and validated solutions approaches to provide start- ups the guidelines and methods to implement open innovation through co-creation with customers. Also, the startups must decide the Require- ment Engineering activities that require external collaboration with the freelancers and the level of involvement expected from the freelancers. Authors’ contributions All the authors of this paper have equally contributed to this paper. All authors read and approved the final manuscript. Funding This research received no external funding. Ethical approval This article does not contain any studies with human participants or animals performed by any of the authors. Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments Our thanks to interviewed startups and freelancers that participated in several interview rounds and shared their unique insights that helped the researchers to complete the empirical study. References [1] V. Gupta, J.M. Fernandez-Crehuet, T. Hanne, Freelancers in the software development process: A systematic mapping study, Processes 8 (10) (2020a) 1215. https://www.mdpi.com/2227-9717/8/10/1215. MDPI. https://www.mdpi.com/2227-9717/8/10/1215 V. Gupta et al. Results in Engineering 8 (2020) 100175 [2] K. Petersen, R. Feldt, S. Mujtaba, M. Mattsson, Systematic mapping studies in software engineering, in: 12th International Conference on Evaluation and Assessment in Software Engineering (EASE) vol. 12, 2008, pp. 1–10. [3] V. Gupta, J.M. Fernandez-Crehuet, C. Gupta, T. Hanne, Freelancing models for fostering innovation and problem solving in software startups: an empirical comparative study, Sustainability (2020b). Submitted for publication. [4] P. Runeson, M. H€ost, Guidelines for conducting and reporting case study research in software engineering, Empir. Software Eng. 14 (2) (2008) 131, 2009. 6 [5] V. Gupta, J.M. Fernandez-Crehuet, T. Hanne, R. Telesko, Requirements engineering in software startups: a systematic mapping study, Appl. Sci. 10 (2020c) 6125. htt ps://www.mdpi.com/2076-3417/10/17/6125. [6] K. Petersen, S. Vakkalanka, L. Kuzniarz, Guidelines for conducting systematic mapping studies in software engineering: an update, Inf. Software Technol. 64 (2015) 1–18. http://refhub.elsevier.com/S2590-1230(20)30081-5/sref2 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref2 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref2 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref2 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref3 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref3 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref3 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref4 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref4 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref4 https://www.mdpi.com/2076-3417/10/17/6125 https://www.mdpi.com/2076-3417/10/17/6125 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref6 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref6 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref6 http://refhub.elsevier.com/S2590-1230(20)30081-5/sref6 Fostering product innovations in software startups through freelancer supported requirement engineering 1. Introduction 2. Related work 2.1. Freelancers in software development: systematic mapping study (study 1) 2.2. Freelancing models of software startups: empirical study (study 2) 2.3. Requirement engineering in software startups: systematic mapping study (study 3) 3. Freelancers in requirement engineering: comparative analysis of empirical studies 4. Case study: involving freelancers in requirement engineering 4.1. RQ. 1 how do the startups involve freelancers during requirement engineering activities? 4.2. RQ. 2 what challenges and benefits do the startups observed due to associations with the freelancers? 4.3. RQ. 3 what are the challenges to continuously innovate the software product unlike the established companies? 5. Overall discussion 6. Future directions of the research work 7. Conclusion Authors’ contributions Funding Ethical approval Declaration of competing interest Acknowledgments References