Note: This article only applies to hybrid customers (those who use both Classic and the new Studio). Classic-only customers will not see these changes reflected in Classic Studio reports.

As of September 2023, we have completed some small adjustments to how embedded content is reported. Before, we found some numbers were over or under-reported if a campaign included one or more links to other campaigns (such as a "newsletter" style campaign). Exactly what has changed for embedded content is described below.

Analyze Update 2023

September 2023 marks the final deployment of changes to how some metrics in Analyze were being reported. The changes only apply to campaigns that had embedded content (e.g. newsletters) and only if the campaign was delivered and interacted with via email.

Note: The changes listed in this article are retroactive and apply to all historical events. In other words, if you have previously pulled data on campaigns with embedded content and try to pull the data again, the numbers will have shifted - even if the report timeframe is in the past.

Which Reports Are Affected?

The following standard reports are now up to date with the corrected numbers. Keep in mind that numbers will only shift for the metrics outlined as new or updated above, and only if you are looking at data related to an embedded campaign delivered via email.

Core reports

• Community Overview (formerly Content Overview) 
  • Centralized reporting for community health for initiatives, viewers, and topics
  • Replacing Executive Summary, Initiatives Overview, and Viewers Overview reports
  • Renamed from Content Overview
• Campaign Performance 
  • Content and user funnel metrics
  • Renamed from Campaign Performance [New Studio Campaigns]
• Campaign Delivery (formerly Campaign Delivery Details)
  • Channel delivery funnel metrics
  • Renamed from [New Studio] Campaign Delivery Details

Updated Reports

• Publishers Overview
• Topics Overview
• Content Amplifier Overview (if enabled)
• Post Performance

New Report

• Email Campaigns Overview

No Change

• Advocacy Insight
• Audience Lifecycle
• Comments Overview
• Video Performance
• Video Overview

Sunset

The following reports will be sunset.

• Article Performance - Use Campaign Performance
• Articles Overview - Use Community Overview
• Executive Summary - Use Community Overview
• Initiatives Overview - Use Community Overview, then view by initiative and filter by initiative
• Topic Performance - Use Community Overview, then filter by topic 
• Viewers Overview - Use Community Overview

Metric Terms

New

Reached Users (replacing Viewers) - Reached users who had an impression, open, or engagement with at least one piece of content. Reached Users gives you a good measurement of your platform's active users.

Impression (replacing View) - A touchpoint for a given user to be exposed to a piece of content, including seeing a content card in web experience, email, Assistant, or push, or explicitly opening an email. Impressions replace Views to distinguish the action from a user actually opening a piece of content and reading it.

Note: To be counted as an impression in mobile, the content card will need to be in view for at least 3 seconds. However, there is no minimum threshold for web. As long as the card is in view in the browser, an impression is counted for web.

Includes:

• Viewing a content card in web, mobile, content amplifier, or microapp (feed, suggested results, search, etc.)
• Opening an email or push notification
• Viewing a content card embedded in another campaign

Excludes:

• When content is loaded on a page but never in 'view panel' of user

Example 1

An impression consists of previewing the content card in the web experience. An open is calculated when the content card is clicked on to open the content in its full view.

Example 2

An impression consists of previewing the content card in the mobile app. An open is calculated when the content card is tapped on to open the content in its full view.

Example 3

Both an impression and an open are calculated when the email is opened. Opening the email allows the user to preview the content, which is calculated as an impression.

Preview - A user's action to generate an impression, such as seeing a content card in the web experience, email, Assistant, or push, or explicitly opening an email. Used interchangeably with Impressions.

Unique Impressions/Opens/Engagements - Represents the number of unique users who interact with a given piece of content. For instance, an article opened by the same user twice will contribute only one time to the unique opens count. 

Updated

% Opened Users - A measurement of how many users active in your platform are reading your content. Renamed from % Opened. Calculated as Opened Users divided by Reached Users.

% Liked Users - A measurement of how many active users in your platform are liking your content. Calculated by Liked Users divided by Reached Users.

Engagements - Actions beyond previewing, opening, or reading a piece of content.

Includes:

• Liking a post
• Commenting on a post or replying to a comment
• Bookmarking a post
• Sharing a post
• Clicking on an embedded link or embedded post within the post
• Playing a video within the post
• Acknowledging a post

Engagement Rate - The Engagement Rate is now calculated as the number of unique engaged users divided by the number of reached users (users with an impression). Engagements are defined above.

Clicks - Refers to a user clicking on a link (content or external link) within a post/campaign.

Includes:

• Clicking a link within the content

Excludes:

• Clicking unsubscribe from an email link
• Opening the content/campaign
• Clicking a link outside of the content
• “Other engagement” such as liking, commenting, sharing, and bookmarking
• Opening an email, particularly when content is opened via a link embedded in another piece of content

Email Open Rate - Calculated now as Emails Opened divided by Emails Delivered.

Email Click-Through Rate - Calculated now as Emails Clicked divided by Emails Delivered. Clicks do not include opening an email.

Emails Delivered - A measurement of how many emails sent via a campaign are successfully delivered to the receiving server, and waiting for users to open them. 

Push Open Rate - A measurement of how many delivered push notifications were opened by users, leading to a redirect to the mobile app. 

Obsolete

Clicked % - Replaced by % Opened Users as top-line KPI

Viewers - Replaced by Reached Users

Views - Replaced by Preview and Impression 

Funnels

Content Funnel

Measures how well a piece of content is digested by users. Contents are delivery channel agnostic.

• Centered around a given piece of content (article, newsletter, media, etc.)
• Same sequence regardless of how it is delivered (i.e. via email, push, etc.)
• Sent -> Previewed -> Opened -> Engaged
• Basic unit - interaction between a user and a piece of content
• Use Campaign Performance report

Channel Delivery Funnel

Measures content delivery efficiency for the given channel.

• Funnel stages will be different depending on the specific delivery channel (i.e. email, push, web feed, mobile, Assistant, MS teams, etc.)
• Basic unit - actions taken, sends, opens, clicks, etc.
• Use Campaign Delivery report or stats in direct delivery channels

User Engagement Funnel

Measures how the Firstup solution drives more engaged employees.

• Centered around a given employee (unique user)
• Basic unit - Unique user
• Use Campaign Performance and Community Overview reports

How Have Metrics Changed?

• Opens may look higher or lower depending on the scenario. In one scenario, opens were under-reported (Scenario 1); in another, opens were over-reported (Scenario 4).
• Email deliveries may look a little lower, as they were previously over-reported for embedded campaigns. (Scenario 2)
• Reached Users may look a little higher, as they were previously under-reported for embedded campaigns. (Scenario 3)
• Metrics related to viewers, deliveries, opens, and clicks (such as engagement rate or activity) will also adjust when the underlying impressions/deliveries/opens/clicks update.

Scenarios

In all four scenarios, we are talking about campaigns that have links to other campaigns. The differences are due to whether we are looking at "parent" campaigns, "child" campaigns, or campaigns created in the Classic Studio interface.

The updates described in this article do not apply to campaigns that have no links to other campaigns.

Scenario 1: Parent Campaigns Missing Clicks

Clicks on embedded content were missing from the parent campaign's report. Clicks are now reported correctly. Clicks may go up in parent campaign reports.

Conditions:

• The parent campaign was created via Creator Studio.
• The embedded content could have been created in either Classic Studio or Creator Studio.
• The parent campaign was delivered via email.
• You are looking at a report for the parent content.

Example for Scenario 1

A user opens Creator Studio email called ‘My Parent Article’ and clicks on the embedded link Leadership Update. It does not matter if the embedded campaign was created in Creator Studio or Classic Studio.

(a) Before the updates, the ‘My Parent Article’ report would not reflect the click on the embedded content.

(b) The click was correctly represented on the report for the embedded content ‘Leadership Update’. 

Scenario 2: Embedded "Child" Campaigns Have Too Many Deliveries

Deliveries were over-reported on the embedded ("child") content's report. Deliveries are now reported correctly. Delivery numbers may go down in embedded ("child") campaign reports.

Conditions:

• The parent campaign was created via Creator Studio.
• The embedded content could have been created in either Classic Studio or Creator Studio.
• The parent campaign was delivered via email.
• You are looking at a report for the embedded content.

Example for Scenario 2

A user is sent a Creator Studio email called ‘My Parent Article’ with embedded content, e.g. Leadership Update. The 'My Parent Article' is successfully delivered to the user and shown as a delivery on the parent campaign's report. Before the updates, a delivery would also be reported for the embedded ‘Leadership Update’ even though it was not sent as a standalone email.

The over-report of delivery for the embedded content would occur whether the user opened the parent email or not.

Scenario 3: Embedded "Child" Campaigns Missing Viewers

Viewers were under-reported on the embedded ("child") content's report. Viewers are now reported correctly. Viewers may go up in embedded ("child") campaign reports.

Conditions:

• The parent campaign was created via Creator Studio.
• The embedded content could have been created in either Classic Studio or Creator Studio.
• The parent campaign was delivered via email.
• You are looking at a report for the embedded content.

Example for Scenario 3

A user is sent an email with embedded content and opens the email. The user opens the ‘My Parent Article’ email and sees the ‘Leadership Update’ as an embedded link in the email. Before the update, Creator Studio users would be missing an impression for the embedded content, e.g. Leadership Update when the parent Creator Studio 'My Parent Article' was opened via email.

Scenario 4: "New" Campaign (Workflows) Delivered via Classic Studio Email

Before the update, opens were over-reported for campaigns created in Creator Studio and delivered using a Classic Studio email. This includes Workflows from Creator Studio. Opens are now reported correctly. Opens may go down in "new" campaign reports if you have been using Classic newsletters and digests.

Conditions:

• The campaign was created in Creator Studio.
• The email used to deliver the campaign was a newsletter, digest, or other "content" email. created in Classic Studio or through Workflows in Creator Studio.
• You are looking at a report for the "new" campaign.

Example for Scenario 4

User opens Classic Studio ‘Weekly Newsletter’ email. Inside this email is a ‘Snow Day Safety Announcement’ that was created in Creator Studio. Before the update, opening the newsletter triggered a click for the embedded content even though it had not been clicked to open yet.

Classic Studio/Workflows Note

Anywhere in this article that we refer to campaigns in Classic Studio, we mean one of two things:

1. You log in to studio.socialchorus.com (Classic Studio) and create an "Email Campaign".
2. You log in to studio.onfirstup.com (Creator Studio), navigate to Configure > Workflows, and create a new Workflow.

Currently, Workflows in the new Creator Studio is a Classic Studio interface. Therefore, even if you have never signed in to Classic Studio directly, you may still have some campaigns created or managed in the Classic Studio interface.

This screenshot shows what Workflows look like in Creator Studio. Any campaign created or managed in this interface would have triggered Scenario 4.

Flowchart

The flowchart below highlights the expected differences in updated reports. To download the flowchart, see the attached file at the bottom of this article.

FAQ

Is the metrics update release for both Creator Studio and Classic Studio? 

This release is in effect for Creator Studio customers and hybrid customers (those who use both Classic and the new Studio). Classic-only customers will not see these changes reflected in Classic Studio reports.

What outstanding issues will be addressed by this metrics release? 

This release will address all the known issues listed in this KB article. 

Should I still use the Post Performance report? 

No, you should use the Campaign Performance and Campaign Delivery reports.

Are there any changes to the Engagement Rate? 

The Engagement Rate (% Engagement) remains the same. The Engagement Rate still includes email opens in this release. This means you will still see a high engagement rate value for email-heavy campaigns. 

Though previously a key metric in measuring platform performance, engagement rate does not factor in active users interacting and reading content, and hence we are working on a next-generation index to replace the engagement rate. In the meantime, we encourage clients to track via % Opened Users and Reached users. 

Engagement Rate will no longer be promoted as the top left metric in key reports such as the Campaign Performance report. 

What is the definition of Viewers after the change?

Reached Users will replace Viewers. Refer to the Metric Terms section. 

What is the definition of Open Rate? 

We no longer use Open Rate as a term. Make sure to distinguish between Email Open Rate and % Opened Users. Refer to the Metric Terms section. 

Why does Open Rate show one value for the Campaign Performance report but another value for the Campaign Delivery report? 

This question was asked before we updated our reports. Now there are two metrics around opens - Email Open Rate vs. % Opened Users. The former refers to how many delivered emails are opened, whereas the latter refers to how many users opened and read your content. You should not confuse these two metrics as they belong to different funnels.

How does the change (Clicks vs. Opens) impact videos? What if someone watches a video in their feed? Is that a Click or an Open?

Video play metrics will not be affected. Any click that opens the content in another platform is a Click. For example, clicks to play a video in an email would open that content in the web experience to play the video. It would technically also be an Open since you are now opening that content in the web experience. The click would be preceded by the email open. Similar behavior could happen within a microapp. For a video in the feed, clicking on the content card would be an Open. Clicking to play the video would be considered a "video play", not a Click.

Where can I find both the percentage of viewers who clicked at least one post and the average number of posts that viewers clicked on in the selected time frame?

This can now be found under % Opened Users in the Community Overview report.

Which reports should I use for statistics on individual/specific campaigns vs. a collection of campaigns?

Both the Campaign Performance and Campaign Delivery Details reports have been updated to allow filtering for a specific campaign only. This excludes any statistics related to content embedded in other campaigns. To view overall statistics for content across all related campaigns, use the Community Overview report and filter by the content as before.

Where Are They Now?

Articles Overview report - The following metrics can be found in the Community Overview report (renamed from Content Overview report).

• Number of posts (Articles) - Replaced by Posts with Views
• Viewers -Replaced by Reached Users
• % Clicked - Replaced by % Opened Users
• Engagement - Information can be found in the Top Posts table and Viewer Details table at the bottom of the Content Overview report
• Usage by Segment - Replaced by Reached Users Breakdown and use ‘view by’ filter to change segment 
• Top Posts - Still available in the Community Overview report (renamed from Content Overview report)
• Usage Details - Replaced by Content Engagement Details

Articles Performance report - The following metrics can be found in the Campaign Performance report and the Video Performance report (for video metrics).

• Viewers - Replaced by Reached Users
• % Engagement - Remains in the updated report
• % Clicked - Replaced by % Opened Users
• % Liked - Renamed to % Liked Users
• Viewer Details - Replaced by Reached Users
• Embedded Videos, Attachments, Links - 
• Usage Details
  • Videos - Remains in the updated report
  • Attachments - Metrics can be found in the Video Performance report 

  • Overall metrics Can be found in the Engagement Breakdown table

Executive Summary report- The following metrics can be found in Community Overview report (renamed from Content Overview report).

• Viewers - Replaced by Reached Users
• % Clicked - Replaced by % Opened Users
• Avg Clicks / Viewer - Replaced by Avg Posts Opened
• Posts with Views - Remains in updated report
• Usage by Segment - Remains in updated report
• Usage Details - Replaced by Content Engagement Trend and Reached Users Breakdown tables
• Top Posts - Replaced by Engagement Breakdown and Content Engagement Details tables

Initiatives Overview report - The following metrics can be found in Community Overview report (renamed from Content Overview report) by setting the filter ‘View By Initiative’. 

• Viewers - Replaced by Reached Users
• % Clicked - Replaced by % Opened Users
• Avg Posts Viewed - Replaced by Posts with Views
• Avg Posts Clicked - Replaced by Avg Posts Opened
• Usage by Initiative - Replaced by Reached Users Breakdown
• Initiative Usage Details and Initiative Details - Replaced by Engagement Breakdown

Viewers Overview report - The following metrics can be found in the Campaign Performance report and the Community Overview report (renamed from Content Overview report).

• Viewers - Replaced by Reached Users
• % Clicked - Replaced by % Opened Users
• % Liked - Can be found in the Campaign Performance report
• Avg Posts Viewed - Replaced by Avg Posts Opened
• Usage by Segment - Replaced by Reached Users Breakdown
• Usage Details - Replaced by Engagement Breakdown and Content Engagement Details
• Viewer Details - Remains in updated report

Topic Performance report- The following metrics can be found in the Topic Overview report, Campaign Performance report, and Community Overview report (renamed from Content Overview report).

• Viewers - Replaced by Reached Users
• % Engagement - Remains in Campaign Performance report
• Posts with Activity - Replaced by Posts with Views
• Usage by Segment - Replaced by Reached Users Breakdown
• Segment Details - Replaced by Engagement Breakdown
• Viewers Details - Replaced by Reached Users Details
• Topic Follower Trend - Replaced by Content Funnel (Bars and Trends)
• Top Posts - Remains in Community Overview report