Text Quality & Style Properties

These properties answer the question: how well-written is the output? They measure stylistic and structural characteristics - whether the output is concise, readable, appropriately formal, informative rather than evasive, and structurally valid.

Information Density

In various LLM use cases such as text generation, question answering, and summarization, it is crucial to assess whether the model's outputs actually convey information and how concisely they do it.

An information-dense output typically consists of readable factual sentences, instructions, or suggestions. In contrast, low information density is characterized by requests for additional information, incomprehensible text, or evasive responses.

The Information Density Score measures how information-dense the LLM output is, ranging from 0 (low density) to 1 (high density).

Examples

Text	Information Density Score
To purchase the Stormlight Archive books, enter the kindle store	0.90
The Stormlight Archive is a series of epic fantasy novels written by Brandon Sanderson	0.87
Wow, so many things can be said about The Stormlight Archive books	0.34
Can you elaborate about the reason you ask so I can provide a better answer?	0.19

Explainability

In the Interactions screen, click on the property while viewing an interaction to see the sentences in the output that received the lowest scores:

Compression Ratio

The Compression Ratio measures how much shorter the output is compared to the input. It is calculated by dividing the size of the input by the size of the output. Useful for summarization tasks where you want to track how aggressively the model is condensing content.

Reading Ease

A score calculated based on the Flesch reading-ease formula. The score typically ranges from 0 (very hard to read, requires intense concentration) to 100 (very easy to read) for English text. Useful for ensuring output readability matches your target audience.

Fluency

The Fluency property scores how "well" the text is written - how close it is to a sample of fluent English text. A value of 0 represents very poorly written text; 1 represents perfectly written English. The property uses a BERT-based model trained on examples of fluent and non-fluent samples.

Examples

Text	Fluency
Natural language processing is an interdisciplinary subfield of linguistics, computer science, and artificial intelligence.	0.97
Pass on what you have learned. Strength, mastery, hmm… but weakness, folly, failure, also. Yes, failure, most of all. The greatest teacher, failure is.	0.75
Whispering dreams, forgotten desires, chaotic thoughts, dance with words, meaning elusive, swirling amidst.	0.2

Formality

The Formality model measures how formal the input text is. A score of 0 represents very informal text; 1 represents very formal text. The model uses the RoBERTa architecture and was trained on the GYAFC corpus.

Examples

Text	Formality
I hope this email finds you well	0.79
I hope this email find you swell	0.28
What's up doc?	0.14

Sentiment

The Sentiment property ranges from -1 to 1 and measures the emotion expressed in a given text, as measured by the TextBlob sentiment analysis model. Useful for tracking whether your application's tone is appropriate and consistent.

Examples

Text	Sentiment
Today was great	0.8
Today was ordinary	-0.25
Today was not great	-0.6

Content Type

The Content Type property indicates the type of the output text. It can be json, sql, or other. The types json or sql mean the output is valid according to the indicated type. Useful for applications that are expected to return structured output and want to catch cases where the model returns plain text instead.

Invalid Links

The Invalid Links property represents the ratio of links in the text that are broken (return a non-200 HTTP status). For text without links, the property always returns 0. Useful for applications that generate content with citations or URLs.