Sentiment analysis

src: mms.businesswire.com

Mining opinions (sometimes known as analytics sentiment or emotions AI ) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, measure, and study affective states and subjective information. Sentiment analyzes are widely applied to voice customer material such as survey reviews and responses, online and social media, and health materials for applications ranging from marketing to customer service to clinical medicine.

In general, sentiment analysis aims to determine the attitudes of a speaker, author, or other subject in relation to some topic or the overall contextual polarity or emotional reactions to documents, interactions, or events. Attitudes can be judgment or evaluation (see assessment theory), affective state (ie, the writer's emotional state or speaker), or the intended emotional communication (ie, the emotional effect intended by the author or a conversationalist).

Video Sentiment analysis

Example

The goals and challenges of sentiment analysis can be demonstrated through several simple examples.

Simple case

• Coronet has the best all day cruise line.
• Bertram has a deep hull V and runs easily through the ocean.
• The 1980s yacht cruise ship from Florida is ugly.
• I do not like old cabin cruisers.

A more challenging example

• I do not like cabin cruisers. (Negative Handling)
• Not liking boats is not my thing. (Negation, reverse word order)
• Sometimes I really hate RIB. (Adverbial modifies sentiment)
• I really like to go out in this weather! (Maybe sarcastic)
• Chris Craft is more handsome than Limestone. (Two brand names, identifying the target of the attitude is difficult).
• Chris Craft is better than Limestone, but Limestone projects procrastination and reliability. (Two attitudes, two brand names).
• The movie is startling with many annoying storylines. (Negative terms are used in a positive sense in a particular domain).
• You should see the decadent dessert menu. (The term attitude has shifted polarity recently in a given domain)
• I like my phone but will not recommend it to any of my colleagues. (Qualified positive sentiments, difficult to categorize)
• The show next week will be true koide9! (Newly minted terms can be highly accomplished but easily changeable in polarity and often out of a known vocabulary.)

Maps Sentiment analysis

Type

The basic task in sentiment analysis is to classify the polarity of the text given in the document, sentence, or level of feature/aspect - whether the opinion expressed in the document, sentence or feature of the entity/aspect is positive, negative or neutral. Furthermore, the classification of sentiments "outside of polarity" is seen, for example, in emotional states such as "angry," "sad," and "happy."

Precursors for sentimental analysis include the General Inquirer, which provides guidance toward the pattern of quantification in the text and, separately, psychological research that examines the psychological state of a person based on the analysis of their verbal behavior.

Furthermore, the methods described in the patents by Volcani and Fogel, look specifically at sentiments and identify individual words and phrases in the text by observing different emotional scales. A current system based on their work, called EffectCheck, presents a synonym that can be used to increase or decrease the level of emotion generated in each scale.

Many other subsequent attempts are less sophisticated, using mere sentiments, from positive to negative, such as Turney's work, and Pang's applying different methods to detect the polarity of their product reviews and movie reviews. This work is at the document level. One can also classify the polarity of documents on a multi-directional scale, which Pang and Snyder tried: Pang and Lee expanded the basic task of classifying positive or negative film reviews to predict star ratings on either one of the 3- or 4-star scales, while Snyder did an in-depth analysis of restaurant reviews, predicting ratings for various aspects of a given restaurant, such as food and atmosphere (on a five-star scale).

The first step of bringing together various approaches - learning, lexical, knowledge-based, etc. - was taken in the 2004 Spring Symposium of AAAI in which linguists, computer scientists, and other interested researchers first aligned interest and proposed joint tasks and benchmark data sets for systematic computational research on influence, attraction, subjectivity, and sentiments in the text.

Although in most statistical classification methods, neutral classes are ignored with the assumption that neutral texts are near binary binary limits, some researchers suggest that, as in any polarity problem, three categories must be identified. In addition, it can be proven that specific classifiers such as Max Entropy and SVM can benefit from introducing neutral classes and increasing the overall accuracy of classification. In principle there are two ways to operate with a neutral class. Well, the algorithm goes on to identify the neutral language first, filter it out, and then rate the rest in terms of positive and negative sentiments, or build a three-way classification in one step. This second approach often involves estimating probability distributions over all categories (eg Bayer naive classification as applied by NLTK). Whether and how to use a neutral class depends on the nature of the data: if the data are clearly grouped into neutral, negative and positive, it makes sense to filter the neutral language and focus on the polarity between positive and negative sentiments. If, on the other hand, the data is largely neutral with minor deviations from positive and negative influences, this strategy will make it more difficult to clearly distinguish between the two poles.

Different methods for determining sentiments are the use of scaling systems in which words commonly associated with negative, neutral, or positive sentiments with them are numbered on a scale of -10 to 10 (most negative to most positive) or from 0 to the upper limit positives such as 4. It is possible to adjust the sentiments of the given term relative to the environment (usually at the sentence level). When a piece of unstructured text is analyzed using natural language processing, each concept in a given environment is scored based on the way words of sentiment relate to related concepts and scores. This allows movement to a more sophisticated sense of sentiment, as it is now possible to adjust the sentiments of the concept relative to the modifications that may surround it. Words, for example, that intensify, relax or negate sentiments expressed by the concept may affect the score. Alternatively, the text can be scored of the strength of positive and negative sentiments if the goal is to determine sentiment in the text rather than the overall polarity and strength of the text.

Identify subjectivity/objectivity

This task is usually defined as classifying a given text (usually a sentence) into one of two classes: objective or subjective. This problem can sometimes be more difficult than the polarity classification. The subjectivity of words and phrases depends on the context and objective documents can contain subjective sentences (eg, news articles citing people's opinions). In addition, as Su mentioned, the result depends largely on the definition of subjectivity used when summing text. However, Pang shows that removing objective sentences from a document before classifying the polarity helps improve performance.

Features/aspect-based

This refers to the determination of opinions or sentiments expressed on various features or aspects of the entity, for example, from a mobile phone, digital camera, or bank. A feature or aspect is an attribute or component of an entity, such as a mobile screen, a service for a restaurant, or a camera image quality. The advantage of feature-based sentiment analysis is the possibility to capture the nuances of an interesting object. Different features can generate different sentiment responses, for example the hotel can have a convenient location, but the food is mediocre. This issue involves several sub-issues, for example, identifying relevant entities, extracting features/aspects, and determining whether opinions are expressed on any feature/aspect positive, negative, or neutral. Automatic feature identification can be done by syntactical method, with topic modeling, or by in-depth learning. A more detailed discussion of the level of sentiment analysis can be found in Liu's work.

src: www.crimsonhexagon.com

Methods and features

The existing approaches to sentiment analysis can be grouped into three main categories: knowledge-based techniques, statistical methods, and hybrid approaches. Knowledge-based techniques classify text by affecting categories based on the presence of unambiguous words such as happy, sad, fearful, and bored. Some knowledge bases not only include clear words of influence, but also give arbitrary words that may be "closeness" to certain emotions. The statistical methods utilize elements from machine learning such as latent semantic analysis, supporting vector machines, "word bags" and "Semantic Orientation - Multiple Target Information". More sophisticated methods try to detect the holders of sentiments (ie, people who maintain affective states) and targets (ie, entities about what effect is perceived). To mine an opinion in context and get a feature on the speaker's opinion, a grammatical grammatical relationship is used. The relation of grammatical dependence is obtained by the deepening of the text. The Hybrid Approach utilizes both machine learning and elements of knowledge representation such as ontology and semantic networks to detect semantically expressed semantics, for example, through conceptual analysis that does not explicitly convey relevant information, but is implicitly associated with other concepts that do so.

Open source software disseminates machine learning, statistics, and natural language processing techniques to automate sentiment analysis on large text collections, including web pages, online news, internet discussion groups, online reviews, web blogs and social media. Knowledge-based systems, on the other hand, make use of publicly available resources, to extract semantic and affective information related to natural language concepts. Sentiment analysis can also be done on visual content, ie images and video. One of the first approaches in this direction is SentiBank utilizing adjectives adjectives representation of visual content. In addition, most of the sentimental classification approaches depend on the bag-of-words model, which ignores context, grammar and even word order. An approach that analyzes sentiment based on how words compose longer term phrases has shown better results, but they incur additional surcharges.

The component of human analysis is required in sentimental analysis, since automated systems are unable to analyze the historical trends of individual commentators, or platforms and are often grouped incorrectly in expressed sentiments. Automation affects about 23% of the comments properly classified by humans. However, humans often disagree, and he argues that inter-human covenants give the upper limit that the classification of automatic sentiments can finally reach.

Sometimes, the structure of sentiments and topics is quite complex. Also, the problem of sentiment analysis is non-monotonous with respect to sentence extension and stop-word substitution (compare THEY will not let my dog ​​stay in this hotel vs. I will not let my dog ​​stay in the hotel this ). To address this problem a number of rule-based and reasoning approaches have been applied to sentiment analysis, including improper logic programming. Also, there are a number of tree traversal rules that are applied to syntactic parse tree to extract the actuality of sentiments in open domain settings.

src: www.win.tue.nl

Evaluation

The accuracy of the sentiment analysis system, in principle, how well it agrees with human judgment. This is usually measured by various measures based on the accuracy and recall of two categories of negative and positive text targets. However, according to the study, human appraisers usually only agree about 80% of the time (see Inter-Assessment reliability). Thus, a program that achieves 70% accuracy in classifying sentiments is performed almost the same as humans, although the accuracy may not sound impressive. If a program is "true" 100% of the time, humans will still disagree with it about 20% of the time, because they disagree about any answer . On the other hand, the computer system will make a very different error from the human appraiser, and thus the numbers are not entirely comparable. For example, the computer system will experience problems with negation, exaggeration, joke, or sarcasm, which is usually easy to handle by human readers: some errors made by computer systems seem too naive to humans. In general, the utility for practical commercial tasks of sentiment analysis as defined in academic research has been questioned, mostly because of a simple one-dimensional model of sentiments from negative to positive results, with little actionable information for clients concerned about the impact of public discourse on such as a brand or company reputation.

To better suit the needs of the market, sentiment analysis analyzes have shifted to more task-based steps, formulated in conjunction with representatives from PR agencies and market research professionals. Focus in eg. the RepLab evaluation data set is less on the content of the text under consideration and more about the influence of questionable text on the brand reputation.

src: i.ytimg.com

Web 2.0

The rise of social media such as blogs and social networks has sparked interest in sentiment analysis. With more reviews, ratings, recommendations and other forms of online expression, online opinion has turned into a kind of virtual currency for businesses that want to market their products, identify new opportunities, and manage their reputation. As businesses seek to automate the noise filtering process, understand the conversation, identify relevant content and follow up properly, many are now looking into the field of sentiment analysis. Further complicating the problem, is the emergence of anonymous social media platforms such as 4chan and Reddit. If web 2.0 is about democratizing publishing, then the next stage of the web may be based on democratizing the data mining of all published content.

One step toward this goal was achieved in the study. Several research teams at universities around the world today focus on understanding the dynamics of sentiment in e-communities through the analysis of sentiments. The CyberEmotions project, for example, has recently identified the role of negative emotions in encouraging social networking discussion.

The problem is most sentiment analysis algorithms use simple terms to express sentiments about a product or service. However, cultural factors, linguistic nuances and different contexts make it very difficult to transform a series of written texts into simple, pro or con sentiments. The fact that humans often disagree on text sentiments illustrates how great a computer task is to get this right. The shorter the text string, the harder it is.

Although a short text string may be a problem, a sentiment analysis in microblogging has shown that Twitter can be seen as a valid online indicator of political sentiment. The political sentiment of Tweets shows a close correspondence with the political position of the parties and politicians, which shows that the content of Twitter messages makes sense reflecting the offline political landscape.

src: i.ytimg.com

Apps in recommendation system

For the recommendation system, sentiment analysis has proven to be a valuable technique. The recommendation system aims to predict the preferences of items from target users. The mainstream recommender system works on an explicit data set. For example, collaborative filtering works on rank matrices, and content-based filtering works on item meta-data.

In many social networking services or e-commerce websites, users can provide text reviews, comments or feedback for items. This user-generated text provides a rich source of user sentiment opinions about many products and items. Potentially, for an item, the text can reveal both the related features/aspects of the item and the user's sentiments on each feature. The features/aspects of the items described in the text play the same role as meta-data in content-based filtering, but the former is more valuable for the recommendation system. Because this feature is widely mentioned by users in their reviews, this feature can be seen as the most important feature that can greatly affect the user experience of the item, while the item's meta-data (usually provided by manufacturers and not from consumers) can ignore features feared by the user. For different items with common features, users can give different sentiments. Also, features of the same item may receive different sentiments from different users. User sentiments on features can be considered as multi-dimensional ranking scores, reflecting their preference for items.

Based on the features/sentiments taken from user generated text, a hybrid recommendation system can be created. There are two types of motivation for recommending candidate items to a user. The first motivation is that candidate items have many common features with items that the user likes, while the second motivation is that candidate items receive high sentiments on its features. For preferred items, it makes sense to believe that items with similar features will have similar functions or utilities. So, these items are also likely to be liked by the user. On the other hand, for shared features of two candidate items, other users can give a positive sentiment to one of them while giving negative sentiment to the other. Obviously, high evaluated items should be recommended to users. Based on these two motivations, a combination of similarity rating scores and sentiment ratings can be constructed for each candidate item.

Except for the difficulties of the sentiment analysis itself, applying sentiment analysis to reviews or feedback also faces the challenge of spam and bias reviews. One-way work is focused on evaluating the help of each review. Poorly written reviews or feedback are hardly helpful for recommendation systems. In addition, reviews can be designed to inhibit the sale of targeted products, making them harmful to the recommendation system even if they are well written.

The researchers also found that the long and short forms of user-generated text should be treated differently. Interesting results indicate that brief form review is sometimes more useful than long form, because it is easier to filter out sounds in the form of short text. For long-form text, the length of the growing text does not necessarily bring about a proportional increase in the number of features or sentiments in the text.

src: mms.businesswire.com

See also

• Market sentiment

src: f.ch9.ms

References

Source of the article : Wikipedia