Mobile Application Development Skills Set Aligned with the E-Cf Framework and Industry Needs

: The number of mobile apps is continuously growing but development organisations are not sure of the good alignment of the skills of application developers with what the industry needs. A precise definition of the professional profile of the Mobile Application Developer (MAD) can help to better understand the needs of software development teams. This cooperation of several European organisations for analysing professional profile of MAD has led to a study of skills and competences resulting in a conceptual model with relevant characteristics: a) based on literature review, b) framed within the European standard e-Competence Framework, e-CF (EN16234) and the ESCO official labour classification and c) empirically validated with qualitative and quantitative data from many stakeholders in the field. This analysis might help to add homogeneity to talent management overcoming possible barriers for international mobility within EU as the concepts are taken from the EU reference models.


INTRODUCTION
The rapid evolution of technology and subsequent changes in businesses continuously increases complexity and dynamism in the IT sector.Constant emergence of new roles and occupations within IT profession stresses the fight for talent and the need of adapting training of engineers to the skills demanded by job market.The development of mobile apps is one of the most productive professional areas as the number of apps has massively grown during recent years, reaching over 230 billion of downloads by 2021 [1].In this scenario, the quality of apps, as well as the need to be developed quickly and with low costs, play an important role in surviving in such a competitive market [2].
The development of mobile applications has specific characteristics (e.g., varied network speeds, network dependency, limited platform performance, varying screen size and resolution, etc.) so app development is considered a different discipline from the development of web/desktop applications [3,4].Those differences hinder the respect to software engineering principles which is farther from what one might expect [5].The development of apps for mobile devices has challenged the development departments, which needed to change and update their practices as projects increasingly involve innovation and have become more challenging: e.g., beyond completely different interface and usability features, apps have shorter life cycles [6] when compared to the one of software for desktop/laptop or websites.They also use a different way of distribution with preinstallation on the device or download from an appstore [2].Besides, the development life cycle of a mobile application moves much faster than the one of a traditional Web application imposing adjustments in the associated life cycle management [7].Clear differences between mobile and conventional development in budget or development time require adaptations of the work with this type of devices [8].In addition, the testing process of these applications is different from the traditional one, mainly due to differences in the user interaction with the device [9].
A survey conducted on the Enterprise Mobility Study [10], where more than 340 respondents from leading global brands participated, highlighted the importance for development teams to focus on business strategy to improve the quality of mobile applications and avoid costly project delays and inefficiencies.Moreover, the same study also considered the lack of marketing and business say as important point to get the best possible chance of succeeding in app stores.Other relevant barriers for good results include the deficit of skills of developers [11], secure ty issues [11,12] and compatibility issues [13,14].
The analysis of information from different job portals, as Monster or Indeed, shows that companies clearly differentiate the vacancies for MAD from the rest for typical desktop or web application development: this indicates that the companies' needs for MAD positions are different.A study performed by Jia et al. [15] identifies specific differences on the soft skills requested for traditional software and for MAD in job ads.The differences in requirements for app development do not only involve those personal skills but also technical competence in specific and different software development processes, project management or testing skills, even including aspects not strictly confined to the traditional field of software development such as, e.g., product and project management, and communication skills with people from different cultures to develop distributed systems [15].These findings suggest that the professionals in MAD could benefit from a specific mix of some business and application marketing skills as well as specific security-related skills, among others.
However, there is a need of determining more precisely the professional profile of competences, skills and attitudes recommended to successfully develop mobile apps.Even the IT professionalism frameworks and reference models such as e-CF EN16234 [16] have not specifically defined competence profile for the role of app developer, they only consider the one of general software developer [17].When looking at ESCO, the official EU labour classification has a profile of MAD [18] among the total of more than 3000 of all sectors.Although this is a very relevant reference, this profile has two limitations: a) it has not been compared with the opinion of a wide sample of stakeholders of the IT job market, b) it is extremely focused on the mere programmer/coder role, while market tends to require profiles with broader spectrum of skills and responsibilities, assuming the full range of activities involved in MAD.
The absence of a complete profile motivated the joint effort of a wide set of organisations to determine it in a solid way, specifically collecting information from relevant stakeholders from different countries as well as from other relevant sources.The goal of the study was improving the precision and completeness of the description of the professional profile for MAD.This profile will enable a better adjustment of training programs, aligning them to job market needs.At the same time, it will be a basic reference for the general identification and assessment of capacities in candidates and professionals in MAD, specifically focused on EU job market but possibly applicable globally.This article presents the results of such study and the APP-CF framework for MAD resulting from it.
The article is structured as follows.Section 2 reviews the previous related works in literature.Section 3 shows the methodology followed in the study while Section 4 shows the previous work to explore the candidate skills for mobile apps developers.Section 5 shows the results from the survey that are analysed and discussed in Section 6.Finally, Section 7 presents the conclusions.

RELATED WORK
A solid specific study on competences for app development requires a prior analysis of existing literature and reference models to determine the candidate skills and the catalogue of profiles demanded by the labour market in the EU.One relevant point to develop a solid professional profile is to ensure homogeneity in the terminology to avoid problems in the analysis of information.Fortunately, the EU has developed two main European references for the ICT labour market which provide consistency in concepts and terminology: ESCO and e-CF.
The European ESCO labour classification [19] is a multilingual classification that identifies and categorises three inter-related pillars (skills-competences, qualifications and occupations) relevant for the EU labour market and education.The profiles of more than 3000 occupations of its catalogue were created through debate and analysis of data by a group of more than 200 experts during four years.According to ESCO, a Mobile App Developer implements application software for mobile devices, based on the designs provided, using general or specific development tools for device operating systems.There is a wide set of skills and knowledge items allocated to the description of the qualification for this position, some catalogued as essential items and others as optional, for example:  Knowledge items like computer programming, mobile operating systems, integrated development environments or mobile operating systems. Skills like analyse software specifications, debug software, develop software prototype or use software design patterns.
The other relevant and available main reference as a sector-specific implementation of the European Qualifications Framework (EQF) is the last version of e-Competence Framework (e-CF)(popularly known as version 4.0) described by the Standard EN16234-1 [16].This framework represents a common language for competences, skills and proficiency levels and job profiles for ICT professionals.It is the result of many years of work with the contributions of ten experts firstly at the Workshop on ICT Skills of CEN and then by Committee TC428.e-CF became a European standard and was published officially as the standard EN 16234-1 for the first time in 2016.The last version EN16234-1 [16] is the fourth version of the framework including a set of 41 e-competences that enable the description of any role in ICT profession.e-CF is complemented by a set of 30 examples of role profiles described in an additional document [17] published in 2018 (based on the version 2016 of the standard) and then updated as Annex in the standard of 2021.Among those ICT role profiles, we can find the general one of Developer whose main purpose is designing and/or coding components to meet specifications of solutions.This profile requires the ecompetences (from e-CF) B.1.Application Development (level 3).B.2. Component integration (level 2).B.3.Testing.B.5 (level 2).Documentation Production (level 3) and C. 4. Problem Management (level 3).The matching between ESCO and e-CF has been already explored in [20] and has served as a reference for contrasting the opinions gathered from experts and stakeholders to help categorize them within these models.
Apart from these two main models, other references, standards and bodies of knowledge delve more deeply into the specific competences needed for MAD.The National Competency Standards for Mobile Application Developers [21] are written specifications of skills and knowledge competencies for their successful performance at work.United Nations Industrial Development Organization (UNIDO), GIZ and National Vocational & Technical Training Commission (NAVTTC) have collaborated to create these standards.They include a competency map with fourunits: Design Screen Layout, Perform Coding, Perform Testing and, Perform Written/Verbal Communication.Although useful for industry, the map is not aligned with the e-CF and does not provide a direct process to inferlearning outcomes.
The International Association of Web Professionals, IWA/HWG, recognized by the CEN (European Committee for Standardization) as the standardization body for Web skills [22], has defined the general third-generation European ICT professional profiles that are primarily relevant to the Web sector.It is mainly based on the document "European ICT Professional Profiles" (CWA 16458:2012)" [23] and the "E-Competence Framework 3.0" (eC-F 3.0, not the last version) [24][25][26].One of the profiles included there is the one of Mobile Application Developer (WSP-G3-017).According to them, the role of Mobile App Developer is coincidental to an ICT developer that creates/codes application solutions for mobile peripherals and writes application specifications for them aligned to customer requirements.Under this classification and according to ESCO, Mobile App Developers only implement application software for mobile devices based on the designs provided by others, so excluding application design from their tasks.The same happens with architecture design (hardware architecture, mobile operating system, etc.) or business plans among other tasks.All of them are ecompetencies from the area "PLAN" of e-CF.However, several works have highlighted the importance of e-business skills in ICT profession in general, and especially for mobile app creators [27].Therefore, the profile WSP-G3-017 might be useful for large companies but it is unclear the same value for SME where developers must contribute to multiple tasks.Since MAD is market-driven, competences related to business and marketing areas linked to apps development could help to understand the app requirements resulting from business goals or market opportunities.In small companies, these competencies could seem necessary to develop successful apps.Besides, the WSP-G3-017 profile only defines 11 competences and those competences are not aligned with the e-CF standard.
We can extract from our literature review several relevant conclusions:  The two main European references are e-CF and ESCO: they are essential for guaranteeing consistency and homogeneity across EU for framing the role profile for MAD.They provide stable and common references for skills and competences within the huge and fast-evolving world of terms in the ICT labour market. There is not any definitive work, developed and validated with the help of industry and experts, describing this profile of MAD with the competences required for a professional quality performance.
Inspired by these two main ideas, we proceeded to launch our research to determine a MAD role profile, especially focused on SMEs, since these type of companies are the main developers of mobile apps [28,29], as well as the ones that can gain the most business advantage with skilled workforce [30].
We carried out two subsequent phases of study to determine the skills and knowledge recommended for app developers.The first one was a preliminary study of the competencies of the workforce of app developers as detected by SMEs through an industry consultation process.Based on the results of this first phase, the second one was aimed at determining the skills recommended for the MAD professional role according to the opinion of the labour market stakeholders.The results from both phases were compared to identify the gap between the needed competencies and the ones detected in workforce at SMEs.

METHODOLOGY
The methodology of the study starts with the following research questions as expression of our goals:  RQ1: Which skills are reporting MAD SMEs as the ones they demand to mobile application developers? RQ2: Which is the level of MAD skills of employees reported by SMEs?  RQ3: Which is the set of e-competences recommended for MAD according to experts?  RQ4: Is it possible to develop an empirically solid profile for the MAD role, aligned to ESCO and e-CF, based on a relevant number of references and data from labour market stakeholders?
As mentioned above, answering these questions requires a study with two main phases as shown in Fig. 1.The first one was a preliminary study to know the situation of MAD workforce in terms of competencies through an industry consultation process.The results of this study would be the basis for researching in a framework of competencies for the MAD role.The second study was aimed at the goal of determining what skills developers should have according to the opinion of the labour market stakeholders.Tab. 1 summarises the sample of participants in the two surveys.

STUDY OF SKILLS AND COMPETENCES IN MAD WORKFORCE 4.1 Method
Answering question RQ1 and RQ2 requires the collection of information from stakeholders.The work had two main phases:  Firstly, a preliminary review of industry practice through literature mentioning the most common skills in MAD. Then two actions in parallel:  Structured interviews to 13 experts to main areas or domains of skills. Qualitative analysis by experts of 30 apps to determine skills needed for their development. A final survey addressed to ICT SMEs asking about the skills really present in their MAD workforce: the questionnaire was based on results from previous steps.
For the first step, the references [29,[31][32][33][34] provided a first set of most common skills present in the profile of MAD.In addition, we interviewed 13SME experts with more than 5 years of experience in MAD:  Apps developer/designer: 3.  MAD teacher/trainer: 2.  Business/management profile: 2.  Entrepreneur: 4.  Mobile operating system/architecture expert: 1.  Recruiter: 1.
All except one of the experts were male and their country origin was United Kingdom (40%), Italy (27%), Spain (27%) and Croatia (7%).These structured interviews aimed at identifying the areas or domains of skills as well as the most important ones in each of them: resulted in the creation of an initial matrix of skills.In parallel, the study included a qualitative analysis of 30 apps by 18 experienced professionals supported by 6 experts: this analysis helped to determine the skills required for their development.
After analysing, cleaning, and removing duplicates when needed, the final number of identified generic skills was 40.The list was initially divided into three general categories decided by the group of 6 experts who also allocated the skills to each category:  Business skills: this miscellaneous category grouped 14 skills from different types:  3 soft skills like presentation or problem solving. 4 general management skills like project management or risk assessment. 2 legal and compliance skills like IPR (Intellectual property rights). 5 specific business skills like financial management or business development. Marketing skills: 5 skills from social media marketing to mobile app tracking. Technical skills for the different phases of development:  7 skills focused on languages and coding like Java or HTML5. 7 skills for app design like human interaction or architecture. 4 skills for testing and quality, e.g., user testing or bug fixing. 3 for maintenance like technical support and feedback management.

Survey Design and Sample
These results were an initial answer to RQ1.But the list of skills formed the basis of the online questionnaire to also answer RQ2.The questionnaire showed the competences classified according to the main areas that the experts identified in the interviews, to help respondents to properly understand the information: technical, and business and marketing skills.Respondents were asked to confirm the skills required for MAD and the level of those skills in their workforce.
The study team contacted SMEs specialised in MAD through several ICT associations in Spain (ATI) and Italy (AICA) and a Technology Educational Association in UK (NAACE).The sample accumulated 73 valid responses: 37 companies from the UK, 24 from Spain and 12 from Italy.The target market of the companies was uniformly distributed among Business-to-Business (44.44%),Business-to-Consumer (45.83%) with the rest (9.72%) participating in both, B2B and B2C.Most of the companies participating in the study had less than 50 employees and they work in the fields of business management (76.06%),marketing (80.28%) and business development (78.87%), all of them involved in MAD.

Survey Results
As the first step, we checked the reliability of the questionnaire using the Cronbach Alpha coefficient for the total scale and for each considered category of competences: technical and business and marketing.We also used descriptive analysis to know the competences most valued by companies as well as to detect data entry or collection errors.The Cronbach Alpha coefficient for the total scale gave a value of 0.967, while for each of the categories the values ranged between 0.805 and 0.896.The conventional minimum value adopted for Cronbach's alpha coefficient is 0.7 while values equal to or greater than 0.8 indicate good reliability [35], so the questionnaire shows good reliability.
Respondents confirmed the initial list of 40 skills used as basis thus confirming the answer to RQ1.Regarding technical skills, the results of this phase suggested that the mobile app developers showed a good level, with no less than 55% of companies reporting high or very high level for all the technical skills (average of 64.39% of companies) except only three cases: animation (16.6%), data analytics (37.5%) and IOS design (23.2%).In the categories of business and marketing-related skills, there were cases with high percentages of reported low or very low value of skills: e.g., 44.44% of companies reported deficit in legal issues skills and 34.73% in intellectual property rights (although data distribution was not skewed).Other cases of deficit of skills were the ones for financial management (26.39% of companies), sales channel distribution (22.85%), mobile app tracking (20.3%), social media marketing (20.3%) or risk assessment (20%).Fig. 2 shows the percentage for each skills category.In summary, according to the consulted companies, their app developers have a good level of technical skills in general, but they detected a deficit in several business and marketing skills.Moreover, companies considered those skills as very important for mobile app developers according to the results of the interviews and other related works [36][37][38].This could be explained by the fact that mobile app developers are expected to have all the necessary technical skills, but there is usually little supply of non-technical skill training or specific activities for developing them.These results represent an answer to question RQ1 and RQ2.

COMPETENCE FRAMEWORK FOR APP DEVELOPERS
Once completing the phase of analysis of the skills demanded by SMEs and their presence in MAD workforce, the study added a second phase focused on the skills for MAD recommended by a wide variety of stakeholders in the labour market: professionals, experts, recruiters and employers, educators and instructors, etc.The idea was offering a more complete framework supported in a wider sample of stakeholders, not limited to SMEs, which can guide all types of organisations by offering a recommended general MAD professional profile.So, the goal was collecting enough representative data to address RQ3 and RQ4.

Method
There are three main approaches in literature to identify and classify ICT competences [39]: a) Using the theoretical foundations of ICT to identify the competence sets based on the relevance of the competences in an individual's career (and therefore not considering companies' perspective).b) Including the view of practitioners by consulting them.c) Grouping competences into familiar categories easily identifiable by respondents.
Similarly to other previous research which had empirically adapted the e-CF framework [40], we have followed a combined option to exploit the advantages of all approaches.Firstly, the ICT competences were collected and classified based on the literature review and on results from an industry consultation.Then a variety of industry stakeholders confirmed the skills and competences and indicated the most important ones to determine the relations among them by responding to an online survey.The method for confirming the initial classification was a statistical analysis, more specifically a factor analysis.

Preliminary Analysis of Catalogue of Skills and Knowledge
The first step was covered with a complementary literature review to determine the competences needed to develop mobile applications.The collected information served for building the instrument to be used for the validation by experts.Apart from the findings of the prior work about skills shown in Section 4, we added an additional varied set of information sources to determine the initial group of competences which captures the common view of all the stakeholders:  Related literature, industry reports, analysis and sector studies addressing mobile app competences [16,28,29,31,32,33,41].  Structured interviews with 50 experts from companies based in UK, Italy and Spain that created successful mobile apps to explore the most important skills according to their experience.Companies were contacted through the ICT and technology associations (AICA, ATI and NAACE).
The qualitative analysis of the whole set of collected data generated a list of 107 skills.A focus group of experts of the multinational research group of different industry and professional associations from Italy, Belgium, UK and Spain (AICA, ATI, NAACE and Agoria) sorted and cleaned them by removing duplicated items, when needed.The number of items was firstly reduced to 90 after adjusting the original list.
The group also worked in adjusting and grouping the selected skills within the approach of EN16234-1 (e-CF) [16] as official European standard for e-competences.Using this common language provides homogeneity to the work so it could be more easily used in all the countries involved in the study as well as across the rest of Europe.So, the skills were reformulated, grouped and simplified and mapped to e-CF: the initial list was reformulated as a set of 44 elementary items of skills and knowledge, specifically linked to MAD, mapped to 12 e-competences, out of the total of 41 of e-CF.The group mapped the initial list of competences to the e-CF framework adapting them, when possible; otherwise keeping the additional ones determined by the previous research separated.Attitudes and soft skills appear in e-CF as embedded items in the description of all competences, so this criterion was followed for our model too.
Subsequently, we conducted a content validation analysis (before launching the online survey) aimed at confirming the competences with the help of experts using a structured interview.The total number of participants was 15 from Spain, UK, and Italy.All of them with more than 10 years of experience, and with different profiles to ensure sample representativeness: 4 apps developers/designers, 2 mobile app teachers/trainers, 2 business/management professionals, 4 mobile apps focused entrepreneurs, 1 mobile operating system/architecture expert, and 1 IT specialist recruiter.Results confirmed the competences used as a reference in the questionnaire: 100% of interviewees agreed about the e-competences for business and technical areas as well as for marketing, but in this case, they suggested three more items connected to the e-competence D.6 Digital Marketing from e-CF.

Survey Design
After analysing the new items, we concluded that they were all either included in our initial model with a different name or embedded in other e-competences.Therefore, we decided to rename them to improve the customization to the field and consequently the understanding of the questionnaire.Once changed, we carried out the second round of interviews to confirm the changes to reach a consensus on the design.Experts added five soft skills too.Since we decided to follow the approach of e-CF of considering attitudes embedded as part of the competences, we did not specifically include them in the survey questionnaire.So, the solution was to add an open question in the survey design to confirm if respondents recommended more research on this aspect in the future.The analysis finally led to a survey design with 44 elementary skills and knowledge items adapted to e-CF through12 e-competences for grouping them.The survey did not directly show the reference to the e-CF e-competences as the standard is not widely known and using so specific terminology would have created confusion among respondents: as each item was mapped to e-CF competences, the subsequent analysis would enable the direct link of the final framework to e-CF.
The survey was addressed to experts working in the area as well as to all types of stakeholder representatives: e.g., developers, recruiters, and trainers/educators.The survey was implemented online with the EU Survey tool of the European Commission and widely disseminated, especially through the ICT associations as well as public employment and learning services in the UK, Italy, and Spain.At this phase, we invited to participate not only the people in SMEs related to MAD, but also any other experts in the area to increase the sample (compared to the previous phase) as well as to consider all the possible relevant perspectives.

Analysis of Survey Results
The size of the sample was 194 valid responses leading to a margin of error of 7% with a confidence interval of 95%.The sample was not gender balanced as 81.96% were male respondents.This was expected due to the usual low representation of women in the ICT sector, especially in very technical fields like app development.Most respondents have tertiary education (87.63%) and more than 5 years of experience (69.59%).The sample was well dispersed across professional occupations with 43.81% of apps developers/designers and a good balance for the rest of the positions.We conducted an exploratory statistical analysis to detect data collection errors and to check the feasibility of factorial analysis.Subsequently, we examined descriptive statistics.The descriptive analysis did not provide enough evidence to eliminate any item without further study.
Once the descriptive analysis was concluded, we proceeded to check the feasibility of factorial analysis.Firstly, we analysed the item-total correlation coefficients to evaluate the extent to which each item is linked to the rest of the items of that dimension, i.e., if they measure the same concept.Additionally, the inter-item correlation was analysed too, to know the influence among the items belonging to the same category.As shown in Tab. 2, item-total correlations were above 0.3 suggesting that each item is related to the rest.Tab. 2 shows the results referring to the 44 elementary items of the survey but mentioning the corresponding skill or knowledge item of e-CF mapped to it for the sake of a short clear reference.
Moreover, validity and internal consistency or reliability of the questionnaire was calculated too.The reliability is the degree of consistency of the scale, and it confirms (or not) if the observed variable measures the real value and is free of error.Reliability is commonly calculated using Cronbach's alpha coefficient.Cronbach's alpha was calculated for the total questionnaire and each dimension.As shown in Tab. 2, the final Cronbach's alpha coefficient values for each dimension and total scale range from 0.779 to 0.925 (Cronbach's alpha > 0.7) showing that the scale is consistent [35].None of the Cronbach's alpha coefficient values were increased by removing any item of the set.
Furthermore, construct validity is the degree to which the instrument measures the construct.Construct validity was tested by using Exploratory Factorial Analysis (EFA).The Exploratory Factorial Analysis (EFA) allows examining the relationships among the variables (in this case knowledge and skills) grouping them as components or dimensions (in this case competence areas) conceptually identified according to the weights of these variables on the component.This analysis will allow to confirm if the adjustment to the e-CF framework was well defined.Before executing EFA, we checked the conditions for applying it: both the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett's Test of Sphericity.Tab. 2 shows the final underlying dimensions between measured variables as well as the coefficients for reliability and construct validity.As shown in Tab. 2, the analysis resulted in high KMO statistics ranging from 0.790 to 0.904 [35] and a significant probability level (p < 0.001) for Bartlett's test.Therefore, we can conclude the applicability of factor analysis is confirmed.Once we confirmed the applicability of EFA, we proceeded to apply it.Since the aim was to reduce the competences to the most important ones, we used Principal Components with Varimax rotation as an extraction method.Tab. 3 shows the final framework called APP-CF.The first column shows the competence areas validated by experts; the next columns correspond to areas, e-competences, and skills and knowledge adapted from e-CF or created according to the research carried out.Note that competence areas of APP-CF correspond with e-CF areas: App business with "A.PLAN", Mobile App Development and Mobile App Deployment with, "B.BUILD" and "C.RUN", and Channel Management with "D.ENABLE".

DISCUSSION
The first phase of our study collected and analysed the information provided by MAD SMEs during the industry consultation process.The analysis of skills demanded by SMEs (RQ1) served to create a basis for analysing the perception of these companies regarding the presence of those skills in their employees (RQ2).Data analysis showed that there is a gap among the skills requested by SMEs according to literature, opinion and practice in app development and the ones shown by employees at the time of the study.The information reveals that app developers tend to have high technical knowledge and skills, but a deficit of competences related to marketing and business which are needed in this area.This pilot study has some limitations.It is an exploratory work what makes our results merely descriptive, so it needs to be confirmed by a larger sample.Also, the skill set used in the questionnaire is not based on standards, something that limits its transferability to different contexts and countries.Despite these limitations, the first phase of our study provides a general insight into the situation of MAD in SMEs confirming what other authors and reports had previously detected.
The second phase was aimed at researching the possibility of determining a common general set of competences for mobile app developers.After collecting and analysing information from stakeholders, it was possible to confirm a high degree of consensus on the set of skills which are most recommendable for the effective and productive performance of MAD.The quantitative and qualitative analysis of data has confirmed a solid determination of the list of skills and knowledge which are recommended for a profile of app developer.This set of items represents an answer to RQ3.This idea is also applicable to other existing tools related to self-assessment of competences based on e-CF (e.g.[42]).
Moreover, the construction of the framework APP-CF, based on a sound statistical method which enables grouping of items together with the map to the e-CF standard [16], with the collaboration of the industry and validated by experts, allows an affirmative answer to RQ4.The value of this framework linked to a European standard resides in its capacity of enabling easier transferability to stakeholders across Europe as well as its value as basis for an easier matching to learning outcomes, which could be a relevant step for improving syllabus of education and training courses for MAD.Furthermore, the framework can also guide the management of HR in MAD, helping to configure recruitment, internal career paths and configuring organisational training oriented to skilling for specific positions.
The findings of the industrial research of this work underline the importance of non-technical skills related to marketing or business for the development of mobile applications.In particular, the greatest deficits are found in user support (managing feedback of clients and communicate with end-users about issues), elaborate quality plans, knowing the legal and regulatory landscape, knowing the risk and opportunity assessment techniques and security issues.Note that some of these deficits are consistent with those reported in other works, such as user support [43] or security [10,12,43,44].
We are aware that our study has some limitations.Although the sample is varied and relevant both in profiles of experts and country origin, more information would be recommendable to confirm the details on competences, skills, and knowledge.Getting information from other European countries and performing a detailed analysis of a good sample of job ads to study how the recruiters express their requirements when trying to hire new professionals for app development would be a necessary additional step in the future.This could be possible through the tool OVATE from CEDEFOP [45] that collects information from millions of Online Job Ads (OJA) and maps them to occupations and skills and knowledge items of ESCO, the EU labour classification.In order to facilitate this work, we have been also working in the mapping of the elementary items of APP-CF to similar items in the catalogue of more than 13,000 skills and knowledge items of ESCO.As a first result, the 44 items of the framework have practically identical equivalent items in ESCO in 20 cases: e.g., there is exactly one "Design user interface" and another "Use software design patters" skills in ESCO.In fact, experts at the beginning of the process for the framework also get inspiration from ESCO when determining the first proposals of items for the MAD profile.Another 18 items have similar equivalents in ESCO, sometimes they refer to broader concepts and in others there are not big differences in description or approach: e.g., "Understand the legal & regulatory landscape" in APP-CF can be linked to "Comply with legal regulations" which is a broader skill, which includes understanding as first step.Finally, there are six items where we can find more or less equivalent items in ESCO, but sometimes they are skills instead of knowledge or vice versa or simply they have some differences: e.g., knowledge "measurement of users experience" in APP-CF has some link with the ESCO skill "measure software usability".
Additionally, our previous research [46] showed the importance of soft-skills and attitudes in the employability of engineers.In fact, the last version of EN 16234 has already added an annex with the first approach to behavioural skills.In the case of the ESCO classification, the section of cross-sectoral transversal skills directly includes a specific subsection on attitudes and values at work in its structure.So, as future work, we will explore this link while collecting more information from stakeholders for future refinement of the framework.The work for a model for matching and bridging ESCO and e-CF [20] should serve as a foundation for the integration coming from the different sources.The approach of combining extensive consultation to sectoral stakeholders with mapping to the relevant skills frameworks like ESCO or e-CF has already allowed the determination of other recommended professional profiles, e.g., for IoT implementation [47].

CONCLUSIONS AND FURTHER WORK
We have defined APP-CF, an e-competence framework for the field of MAD after a complete process of analysis of existing studies and references as well as after a detailed collection of information from a very relevant sample of stakeholders and experts.It is mapped to e-CF to ensure transferability and validity in the future.This framework can help to derive the learning outcomes of training programs to improve the competences of IT curricula in the field of MAD, but it also represents assistance to recruiters and HR managers in the personnel selection processes: of course, it can also guide developers who want to improve their professional profile or move to MAD specialization.Additionally, it would help students and professionals to choose training courses, official or not, to more efficiently improve their competences for MAD.The analysis of the information from candidates and recruiters would help to refine details of the framework to add an extra step of validation and fine-tuning.We are also planning an additional collection of information from experts from other European countries and with the presence of even a wider variety of profiles (e.g., more ICT recruiting experts and ICT managers like CIOs).
Finally, the process followed to get the evidence-based framework APP-CF herein described to also serve as a guide for determining the recommended competences for other professional profiles according to the needs of the industry.It is also valid to improve the employability and bridge the gap between the expectations of the industry and higher education graduate profiles.

Figure 1
Figure 1 Steps of the methodology Table1 Summary of samples of the two surveys of methodology Survey # Participants 1 Skills in MAD SME employees 73 SME representatives 2 Recommended skills 194 MAD experts

Figure 2
Figure 2 Percentage of low or very low levels and high or very high levels of skills for MAD to consulted SMEs

Table 2
Summary of reliability and construct validation results

Table 3
APP-CF.Final framework for mobile app developers