GOAL – ORIENTED CONVERSATIONAL BOT FOR EMPLOYMENT DOMAIN

This paper focuses of the implementation of the goal – oriented chatbot in order to prepare virtual resumes of candidates for job position. In particular the study was devoted to testing the feasibility of using Deep Q Networks (DQN) to prepare an effective chatbot conversation flow with the final system user. The results of the research confirmed that the use of the DQN model in the training of the conversational system allowed to increase the level of success


Introduction
In the recent time, due to significant advances in Natural Language Processing (NLP), researches toward conversational bots has greatly accelerated in many research centers around the world.Their main goal of these researches is to replace people in solving a wide variety of time-consuming and tedious problems in many domains, thereby saving staff time and substantially optimizing business costs.As the one of the most popular development directions for conversational bots, banking customer service support bots (Suhel et al. 2020) should be distinguished, where the bot answers the simplest questions and, in the absence of sources of knowledge, refers to the appropriate specialist.Another important application of bots is support in the medical field related to remote patient consultations.They provide people with the opportunity to receive medical care without having to visit a hospital in person (Bharti et al. 2020, Solanki et al. 2023).Conversation bots are also used in public services to perform journalistic tasks to gather, produce and distribute news (Bronwyn, rhianne 2019).
The conversational bots are used as intelligent assistants for Smart Home solutions as well, where they can support intelligent decision-making, predictive and preventive analysis (Salvi et al. 2019).However, in recent times, the most attention should be paid to the milestone in the development of the NLP domain that are the machine learning models, which are utilized for text recognition, interpretation, and generation of natural language texts.Among these algorithms, the best known and one of the most effective is GPT, pretrained on very large datasets, which can be applied in various fields.However, there are also areas in which it does not apply (thorp 2023).A prime example can be observed is writing scholarly articles, since it lacks specific domain knowledge.
The next case is related to recruiting domain, where GPT fails when creating a virtual resume of an employee.In this case, there is a need to process a lot of information from Internet, inaccessible to the algorithm.Moreover, GPT chat cannot ask questions, since its main purpose is to generate answers.In addition, its data resource is not updated in real time, consequently, it is not able to refer to current data.
This paper focuses on solutions, which optimize the process of searching for employees, which in today's world is a complex task.The first issue, significantly requiring a lot of commitment from recruiters, is a need for specific requirements for the chosen position.The employers are determined to find an employee who will perform his duties professionally.On the other hand, jobseekers based on their experience also have requirements such as job benefits, salary, place of work and since the number of job offers on the market is enormous, it is very hard to find the satisfactory job ads.One of the solutions are websites such as LinkedIn, which collect data about available job opportunities and possessed skills, experience or education of professionals, which can strongly facilitate the

26, 2023
Goal -Oriented Conversational Bot for Employment Domain 113 recruitment process.Nevertheless, at the moment, HR departments are still responsible for the recruitment process and have to spend time finding the right candidates, contacting them and checking their fit for the specific job positions.They absorb a very high level of human and financial resources, since the full process is intricate and highly time-consuming.Recruiters and recruits are often frustrated by the lack of competence, knowledge or requirements for the position.
For this reason, the Emplocity Ltd. developed conversational bot, which greatly facilitates the whole recruitment process -Emplobot (DrozDa et al. 2019).It searches and analyzes the content of job listings on websites and tries to match ideal candidates with available job ads.For this purpose, it develops a virtual profile (virtual CV) of a candidate based on a conversation with the user.To make possible creating such a resume, it is essential to keep the interview as short as possible, so that the candidate does not get bored and abandon the conversation.
Therefore, this paper focuses on providing a solution based on a goal-oriented dialogue system that will be able to prepare a candidate's virtual resume in the fewest possible steps.In particular, in this work, we will propose a solution based on a goal-oriented chatbot using reinforcement learning combined with chatbot dependency graphs (le et al. 2019).This allows for dynamic optimization of dialogue flows.Additionally, negative rewards are used, so that the chatbot learns what actions are detrimental to the flow of the conversation, making its questions more relevant.This creates the ability to dynamically select the best conversation flow based on user behavior and conversation context.This concept significantly reduces the number of questions needed to create a complete virtual profile of a candidate, resulting in more frequent achievement of the goal.The research described in the experimental section showed that by using reinforcement learning algorithms, the number of questions to obtain a virtual resume was reduced from 11 to 7.This allowed to increase the percentage of successful conversations from 11% to 64%.
The rest of the paper is organized as follows.The subsequent section identifies the applicability of goal-oriented chatbots described in the literature.Section 3 presents the methods used.The section 4 consists of an evaluation of the models used along with a description of the results.The final section summarizes the paper.

Related work
Recently, the field of chatbots has gained a significant advantage in terms of scientific research as well as their application in practice (SchauB, vauDapiviz 2019).The first chatbots, which were ordinary dialogue systems, were able to receive data and provide information.One of the main directions in the development of conversational bots and their introduction into industrial Paweł Drozda et al.
applications is the broad field of goal-oriented bots.They have achieved their popularity due to the fact that they are capable, not only having an ordinary conversation with a person, but are able to solve specific real-life problems as well, where the main focus is on achieving a defined goal.
In most cases, deep learning methods, often based on transformer neural networks, have been used to develop the corresponding intelligent models implemented in conversational systems.Such neural dialogue systems can directly interact with a structured database and assist the user in accessing information and performing specific tasks.For example, one of the successful applications can be a booking movie ticket system described in (li et al. 2017).It asks the user goal-oriented questions, where during the conversation, it collects information about the customer's needs and at the end reserves the ticket for the user.At the end of the conversation, the environment evaluates the result whether the movie has been booked and whether it meets the user's requirements.In order to comprehensively train the task-completion dialogue system, the authors proposed using supervised learning and reinforcement learning techniques.The techniques used proved to be more effective and resilient than rule-based agents, additionally enabled natural interactions with users in real time.During their research, the authors made the observation about errors that their impact at the slot level is greater than at the intention level.
Another implementation of goal-oriented conversational bots is the software development, where the interactions with the user are one of the most important issues when performing research on chatbots.Leveraging NLP techniques, they can be a support for young engineers by conversing with the user in text form to elicit requirements, identify goals and software expectations (arruDa et al. 2019), construct domain models from text documents, or detect inconsistencies in requirements (GervaSi, zowGhi 2005).
Specialized application does not limit the capabilities of Chatbots only to the technology industry.It is a solution to common problems such as healthful and balanced diets (praSetyo et al. 2020).It runs on the Google Assistant platform and allows you to set goals and guide the behavior of users toward achieving goals.Its main task is to receive information from the user and organize the data accordingly.In addition, it encourages users to achieve their goals with personalized, practical feedback.The achievement was to eliminate commonly noticed design flaws of popular mobile health apps.
The next application of goal-oriented bots is the use of algorithms from the sentiment analysis domain.In the paper (Murali et al. 2020), the authors rise to the challenge by being sensitive to user comments and conversation tone through the implementation of a fine-grained sentiment analysis module (Murali et al. 2020).The obtained results in the experimental session achieved a satisfactory f1-score of 0.79 for the NER system.For the sentiment analysis system, a kappa score of 0.90 for six sentiment classes was reached.Finally, the toxic comment classifier achieved an average accuracy of 92.50% which, in addition to its main purpose, allowed to identify sentences that did not contain vulgarities.
The goal-oriented conversational systems can serve as a receptionist who can answer the most common questions, which successful implementation can be found even in a large institution like a university (kuMar et al. 2018).The system is developed on the basis of two models that implement recurrent neural networks.The first of the deployed solutions allows preparing the right answers to questions about the facilities within the university.The second is a generative model, which is able to answer all other questions provided by the first model.The results of the model evaluation are the following: the first model had an average answer accuracy of 92, while the second model obtained an average BLEU score of 0.53.
Next research was proposed in (lone et al. 2022), where the authors used the SEQ2SEQ architecture to create the chatbot, which uses a reinforcement learning algorithm to respond to user queries.The main goal was to prepare a model, which can prepare answers in an engaging and interactive way, responding to people's emotions.The model does not just use items from a ready-made database, but it is able to learn new answers and queries on its own.According to the authors, the model has demonstrated remarkable accuracy.
It is worth noting that goal oriented chatbots also were successfully applied in relatively new technologies such as blockchain (QaSSe et al. 2021).The main problem regarding blockchain is the necessity to complete complicated tasks during process to obtain smart contract agreement.Smart contracts can be established in an incremental and fully interactive manner.Based on the user's preferences and the performed transformation of the model to text, it is able to generate the code of the smart contract.In order to create a conversational chatbot, the authors used the framework bot Xatkit.They developed a Unified Reference Model for smart contracts (haMDaQa et al. 2020).User intent is detected based on a predefined set of expressions.
Goal oriented chatbots were also introduced in the field of finding the evidence of fraud (liu et al. 2020(liu et al. , SerBan et al. 2017)).In particular, their main task is to support security officers in countering financial fraud.In the paper (liu et al. 2020), the authors proposed a hierarchical reinforcement learning method, which learns high-level rules to guide the conversation, while learning low-level rules are used to generate responses according to predetermined guidelines.In addition, each dialogue has a binary label indicating whether the goal has been achieved.GoChat is able to generate high-level utterances and features proactivity in conversation.The experimental session allowed to obtain the following results: BLEU 9.70, distinct-1 0.061, distinct-2 0.479.In addition, user tests were conducted, which yielded satisfactory results.

Paweł Drozda et al.
Although the research reported covers a great range of applications of goaloriented bots in a variety of fields, the use of this technology has not been addressed in the recruitment problem.To the best of the authors' knowledge, this paper describes the first attempts to implement goal oriented conversational bots in the field of hiring, where, with their use, the process of obtaining a candidate's virtual resume was significantly shortened, and the number of abandoned conversations by job seekers was reduced.

Methodology
The main objective of the research described in this paper was to develop a chatbot conversation method based on a goal-oriented strategy to provide an effective dialog between user and bot.The effectiveness is measured by the percent of conversations that achieves the goal, which in the case of the research conducted is an invitation for a position from a recruiter.To achieve this aim, it is necessary to gather sufficient information about a job candidate with the least number of questions.It will cause the reduction in the number of conversations abandoned by users.

Conversation flow
As part of the work on the conversation flow, the main goal (success) was defined first as: -gathering enough information about the candidate, but so that he or she does not abandon the conversation; -acceptance by the recruiter of the prepared virtual resume and invitation of the candidate by the recruiter to the next stage.
In order to achieve the intended goal, the chatbot had to ask the user a series of questions to gather the necessary information.The general flow of the conversation is illustrated in the Figure 1.
After saying hello, the bot asks the required questions that the user must necessarily answer.Among these questions are first name, last name, agreement to RODO and the last position at work.Without an answer, the conversation is terminated.The second part of the questions is crucial information, however, in the final conversation it is not necessary to get all of them answered.After getting the necessary answers, the bot finalizes the conversation.In the case of the first conversation runs without using the DQN learned network model, where the chatbot asked most of the questions, the sample conversation run was as depicted on Figure 2. In this case, for example, when recruiting for simple jobs, many of the questions asked, despite their high value in creating a virtual resume, were unnecessary and often caused the conversation to be abandoned.This allowed us to optimize the designed flow using a recommender, which responded to subsequent questions and answers indicating the best paths to achieve the desired goal.To optimize the conversation flow, Deep Q Network(DQN) was used.A rigid flow was removed and both the length of the questions and the length of the entire flow were altered depending on what data the candidate was providing and what position they wanted to apply for.With 3,150 pre-approved profiles by recruiters, we were able to train the network with reinforcement by simulating agents to ask the questions needed for the position.At each stage of the dialogue between bot and human, a risk function was established, which determined the probability of abandoning the conversation, and a success function, indicating the level of completeness of the necessary data obtained from the user.If the risk increased above the predetermined rate and the success function reached the assumed value, the system finalized the conversation.
By using DQNs, it was possible to significantly shorten the conversation in many cases and increase the conversational effectiveness of Emplobot.A detailed description of the methods used is presented in the next subsection.After the network was learned, an example of the conversation flow was presented in the Figure 3.A very high variability in the path of the interview was seen for recruiting for simple and specialized positions.For example: -for Courier -the last position is sufficient (for example: unemployed/student); -for Python Developer -education, experience, previous positions, English level, detailed skills, Python experience; -for Private Capital Depositary Specialist with French -French level and education; -Junior Post-Sales Service Specialist -experience, employment history.

Methods
The proposed methodology was inspired by the methods that were applied to the problem of booking tickets at the cinema through a chatbot conversation with the user (li et al. 2017).
The knowledge base for training models in the conversation system contained 3,150 dialogues and was collected using the basic version of Emplobot.The final dataset consists of the candidate profiles (in the form of a set of slot: value) accepted by recruiters in real world scenario.In addition to the annotation of candidate profile information within Emplobot, user intentions were also annotated, supporting training, but not affecting goal achievement.The predefined slot categories and possible intentions are shown in the Table 1.A diagram of the conversation system is shown in the Figure 4.In order to provide comprehensive training for this dialog system, a chatter simulator was implemented for automatic and natural interaction with the dialog system.In the dialogue setting, the user simulator first generates a user goal (acceptance of the profile filled by the simulator by the recruiter by filling the appropriate number of slots).Then to achieve the goal, it generates consecutive questions (chatter actions) to fill the appropriate slots.The answers to the simulated agent's questions by the system in the initial iterations are randomly drawn from the available knowledge base accumulated by the project (100k professions, more than 50k skills, as well as education level, language proficiency, etc.) and thus fill the slots needed to create a virtual resume.
Paweł Drozda et al.In subsequent iterations, the network DQN adjusts the length of the conversation and the number of questions, which becomes more focused around the target as part of the learning.This allows the conversation to be significantly shortened and the goal to be achieved more efficiently.

Experiments
The research evaluated methods based on reinforcement learning, where the work consisted of training the DQN model.For this purpose, the training dataset was previously prepared, more precisely described in the previous section.The research consisted in determining the effectiveness of the obtained model in the context of the previously defined goal, i.e. accepting the resume and inviting the candidate to the next stage of recruitment after the shortest possible conversation with the chatbot.
The input dataset consisted of 3,150 conversations previously conducted in the basic version of the chatbot and was divided in proportions of 70% and 30% into a training and test set.The experiments consisted of training the DQN network through successive iterations of chatter simulator conversations with the chatbot, where in each iteration meant 100 conversations.The percentage of goal achievement was then determined by indicating positive candidate verifications based on a test set.The results are presented in Table 2.As can be observed, in the initial iterations, candidate approval oscillated around 10%.By training the network, the success rate increased to 64% after 24,000 iterations.The evaluation of the test dataset show that the system learns to treat the positions of candidates vectorized similarly to what it saw without knowing exactly such a position earlier.
In addition to testing in an experimental environment, the final model was also implemented in a production environment.In this case, data was collected in real cases not using the model and chatbot conversations with real users after the model was implemented.A summary of the results can be found in the Table 3.As could be observed, the real acceptance rate deviated from the level achieved by the simulations of candidate agents.It is possible that this was influenced by factors related to external variables, such as a larger population with the development of the service, a change in the preference of positions in the recruitment market, changes in the approach of recruiters over time, etc. Nonetheless, it should be noted that the use of the trained DQN model allowed the level of target achievement to increase from 15% to 45%.
The work also reported on the average number of questions asked in the DQN learning process.A summary is presented in the Table 4.As in previous cases, the model used in this one also allowed for significant optimization.It should be noted that in the initial phase of learning the model, the conversation consisted of about 11 questions, where for the learned model it averaged just over 7.

Conclusions
The main goal of this paper was to implement goal -oriented chatbot, which will prepare the virtual resume of the candidates for the job position, that will be accepted by recruiter.In particular the conversations of Emplobot in the basic version were taken into account as a knowledge base and the DQN model was trained to optimize conversation flow.
With the trained network model, it was possible to significantly increase the conversation success rate, which was defined as the user finalizing the conversation with the creation of a suitable resume for the position and the recruiter accepting the candidate for further processing.By using the DQN network model, it was possible to increase the success rate from 15% to 45% in a production environment and reduce the average number of questions asked by the chatbot from 11 to 7.

Fig. 1 .
Fig. 1.The general flow of the conversation

Fig. 2 .
Fig. 2. Conversation example without using the DQN network model Fig. 3. Conversation example with the use of the DQN network model

Table 1
Predefined slot categories and possible intentions

Table 2
Results for goal achievement as percentage in experimental environment

Table 3
Results for goal achievement as percentage in production environment Results of the number of questions asked by chatbot during DQN Network Learning