2018-08-27, Pustulka, Elzbieta, Hanne, Thomas, Blumer, Eliane, Frieder, Manuel, Wong, Ka Chun

We are developing a multilingual sentiment analysis solution for a Swiss human resource company working in the gig sector. To examine the feasibility of using machine learning in this context, we carried out three sentiment assignment experiments. As test data we use 963 hand annotated comments made by workers and their employers. Our baseline, machine learning (ML) on Twitter, had an accuracy of 0.77 with the Matthews correlation coefficient (MCC) of 0.32. A hybrid solution, Semantria from Lexalytics, had an accuracy of 0.8 with MCC of 0.42, while a tenfold cross-validation on the gig data yielded the accuracy of 0.87, F1 score 0.91, and MCC 0.65. Our solution did not require language assignment or stemming and used standard ML software. This shows that with more training data and some feature engineering, an industrial strength solution to this problem should be possible.