Announcing Sitecore 9

Sitecore Experience Platform 9.0 was recently released and the product takes a significant step forward in both flexibility and feature set.   I have had early access to a pre-release of the product and here are some of my favorites things that have changed since Sitecore 8.2.

Sitecore Forms

We have utilized Web Forms for Marketers heavily, empowering authors to create forms and capture data but also integrating custom payment features, encryption and custom fields into the product for our clients.   When walking a client through the features of Web Forms for Marketers, it almost always elicits the exact same reactions:

  1. Does it have to look like that?  This is a pretty basic looking form and we want to ensure usability for our users.
  2. When forms get long, can we break the questions up into multiple pages.

The exciting news is that Sitecore Forms addresses both of these complaints.  Learn more here.

MongoDB

When organizations are considering Sitecore, one of the hosting implications is hosting a noSQL database called MongoDB.   This database stores the analytics information about interactions on the site and drives personalization, email experience manager and other marketing automation components.    While there are cloud solutions provided by Sitecore, mLabs and others, many organizations prefer to be in direct control of this type of data.  However, most organizations don’t have the in-house technical expertise to host MongoDB, back it up and keep it healthy.

Sitecore 9 allows organizations like this to use alternative databases to store this type of information.  Utilizing MongoDB, SQL Azure, SQL Server 2016 or even Azure’s DocumentDB, organizations can store their analytics reliably in a technology they feel comfortable with.

Federated Authentication

Given that most of the organizations we work with have Active Directory domains, a common requirement s to set up a way for users to use their Active Directory credentials to log into Sitecore.  This is possible in older versions of Sitecore using a provided module, but there are specific limitations and configuration choices that often mean that this is not an option, especially when hosting in the cloud.

With the move of many organizations to Office 365, Azure Active Directory is now already enabled in many organizations and the changes in Sitecore 9 allow for easy integration with this services using ASP.NET Identity.  This allows for users to log into Sitecore using a common set of credentials without hosting limitations.

Better Marketing Automation

Under the hood, Sitecore 9 has completely overhauled the way that marketing automation works.  Sitecore 9 is now set up to be omni-channel and to take action on data coming from any system using XConnect.

The end result is that the experience to an author and to the end user of a personalization or optimization experience is seamless and powerful.  The interface for marketing automation, now no longer needing Microsoft Silverlight, is slick and easy to use.

Verdict

Overall, Sitecore 9 is a significant update that provides better long-term flexibility to the platform and delivers important features that users have been waiting for.  More to come in the next few weeks.

Google PageSpeed Insights and Sitecore

Google now prioritizes fast pages in the search algorithm and when pages do poorly on these tests, it may be for a wide range of reasons, some content and configuration.  There are some quick wins in Sitecore for both that will help your site perform faster and pass these types of tests.

The first thing to note is that the Google PageSpeed Insights test is checking to see if your obeys some specific rules.  Passing these rules does not mean that users will have a fast experience of the site, but it will help.   It’s also important to look at true page speed metrics when evaluating the health of your Sitecore instance.

The Google PageSpeed Insights uses a variety of different rules to score the page.   Depending on the type of error you are seeing, there may be different solutions needed.

Reduce Server Response Time

The first step in delivering the page to the user is downloading the entire HTML, which Sitecore is generating dynamically on the server.  When the page contains custom logic, complex queries or searches, the page can be returned slowly to the visitor.  This type of issue typically becomes worse the more users that are on the site at the same time.

The solution for this type of error is looking at ways to cache components on the page and looking for ways to rewrite search queries to reduce the time they take.  In some cases, additional hardware may improve response time.   The goal is to have the HTML delivered to the visitor within 200ms of the request.

Image Optimization

Images uploaded to the Sitecore media library are often not optimized for the smallest possible file size.  As pages often contain multiple images, the impact of this is high on the total time it takes the visitor to download the page.    Images like jpgs and pngs can be compressed in a way that does not lose any image quality.   Standard “export for web” options in Photoshop or similar programs do not pass Google’s strict image optimization standards.

We can either do this by working with authors to only upload web optimized images or we can implement an optimization inside Sitecore to ensure that all images get compressed to the smallest possible size.

Dianoga is a free marketplace module for Sitecore that uses jpegtran and PNGOptimizer to reduce the size of these images when they are delivered to visitors.  It does this by modifying the image in the MediaCache to be optimized using these free tools.  The compression is done asynchronously, so there is no degradation of the performance while it is compressed and the image is only compressed on the first time it is requested.

Image Optimizer v2 is a tool on the marketplace that uses the same underlying tools, but instead handles the process in the Media Library.  A button media items and folders allows you to recursively optimize imagery and save actions in Sitecore ensure that new images uploaded are automatically optimized.

Note: Google PageSpeed tests lately have no longer seen images adjusted by jpgetran and PNGOptimizer as optimized.  I’ve taken Image Optimizer v2 and forked it to use an alternative library (Magick.NET).  Read more about that here

Minification

Once the HTML is delivered to the visitor, the browser begins to process that HTML and make subsequent requests to the server for the JavaScript and CSS files that are needed to render the page.  There is significant overhead in downloading these files, so it is more efficient to deliver a single slightly larger file than it is to deliver a larger number of small files.  To address this, Google recommends a “minifcation” process where files are combined strategically and the overall size of them reduced by removing white space and comments.

If Google PageSpeed Insights is suggesting that you “Minify JavaScript” or “Minify CSS”, make sure you are using a front-end build process, such as Webpack, Gulp or Grunt, to combine and minify these files before including them on the site.  This is typically something that you have to set up in the way you develop your site, but if your site does not change frequently, you can look at external tools to minify what you have, though you would need to go through that process each time.

You will likely also see a recommendation to “Minify HMTL” and this one is more complicated.  It is possible to minify the resultant HTML that the server creates, further improving the transfer speed.   There is an example of minifying the HTML with custom logic on Anders Laub Christoffersen’s blog.

This type of change may have negative impacts on the functionality of the page, especially conditional commenting for cross-browser support, so testing should be done to ensure it works properly.

Render Blocking JavaScript

When JavaScript is included in the <head> block of the HTML, the browser makes the assumption that this file is needed in order to begin the rendering process, causing the browser to wait until the download is complete.   Whenever possible, JavaScript should be loaded at the bottom of the page, allowing the browser to process the bulk of the HTML before waiting on JavaScript file delivery.  This leads to more of the requests happening simultaneously.

In some cases, JavaScript must be injected earlier in the page than is ideal.   Some sites include JavaScript logic on the controls themselves and that code requires JQuery to be included on the page in order to run.  Ideally the implementation moves this included JavaScript to a separate file, but if that is not possible, JQuery must be included early.   Web Forms for Marketers also injects JQuery javascript into the page to make the components function and we do not have the ability to control that.

Leverage Browser Caching

When the browser delivers an individual file (image, video, css, js, etc) it indicates to the browser how long it should trust that version of the file before requesting an updated file.  By default, IIS does not provide an expiration date for content, which effectively causes browsers to ask for the file every time.   Google recommends that we cache the values for at least 1 week.   These values can be configured in IIS with instructions found here.

Note: This may mean that deploys that update static files may not be seen by client browsers until their expiration date.  In order to force updates of cached critical files, adjustments should be made to the URL to avoid the cached value.  An example of this approach would be to append a build number or date as a part of the query string when requesting these files.

Keep Testing

Sadly, this is not a single solution but an ongoing battle.  The rules that Google is following are constantly evolving and the features and content of your Sitecore site are constantly changing.  It’s important to keep coming back and testing.

 

Azure Insights: Azure SQL Database Advisor

Starting in Sitecore 8.2, Sitecore now supports Azure SQL, which offers a lot of benefits when it comes to scalability and maintainability of SQL Server in the cloud.   One of the overlooked benefits is the way a cloud application like Azure SQL can monitor and provide real-time feedback on application performance.

After running Azure SQL for a while, I started to receive recommendations from the Azure SQL Database Advisor.

capture

This service from Microsoft is analyzing the queries happening on my Sitecore databases and making recommendations that will improve the performance.  In this particular situation, Azure has discovered that adding an index to my VersionedFields table will have an impact that may be worth looking into.

Now, in talking with Sitecore support, it is worth noting that applying these changes may improve the performance on some queries while negatively impacting them in others.  The product has not been tested in this configuration, but creation of indexes should typically only benefit the application.

Sitecore Stack Exchange

There are many places you can go to engage with the wider community of Sitecore developers.  A recent addition to this list is the Sitecore Stack Exchange.  Sitecore Stack Exchange is like StackOverflow, but specific for the Sitecore community.  Ask your questions and people can answer and vote on answers.

While you’re at it, make sure to check out Sitecore Community and Sitecore Slack

Session: Better Together: Sitecore on Azure

With the move to Sitecore 8.2, it became clear that Sitecore would have more support for Azure.  Today’s session on the developer track made it clear just how much Sitecore will support Azure moving forward.

One of the key focuses in an upcoming version of Sitecore will be full support for Azure Web Apps, which are a Platform as a Service (PaaS) offering.  In order to accomplish this, there have been some key enhancements to Sitecore:

Session State Cache

The session state service is being re-architected to work on Redis Cache.  Since Sitecore defaults to in-role caching, it becomes hard to scale out Sitecore delivery servers in Azure if users cannot transfer session between servers.  Utilizing Redis Cache is a performant way to handle this while also being supported in on-prem configurations that may need this.  This enhancement is due to come to the Sitecore Azure module as well, in an upcoming final release.  Utilizing this feature in Azure reduces the burden on IT to manage and maintain the technology.  It also seems like a great opportunity to expand more of Sitecore’s caching into Redis in the future.

Application Insights

When utilizing Azure Web Apps, access to the file system is not allowed.  The Web App mounts the application files in a read only network share, which makes things like Sitecore logging and performance counters problematic.  Since Azure has a powerful Application Insights feature to track resource utilization and performance issues, Sitecore has been extended to write logging and counters to Application Insights instead.

Azure Search

Traditional on-prem solutions for Sitecore leverage lucene or Apache Solr to index the content and search the data in a performant way.  Azure Search is a cloud-hosted way to index and query data in a way that alleviates the burden of management from your IT team.  Azure Search will crawl Sitecore content and allow for complex searches, including geo-spatial queries.

SQL Server Georeplication

While Sitecore 8.2 was the first version to support Azure SQL, the application logic of Sitecore does not allow for a secondary set of databases that are required in SQL Georeplication.  The upcoming release of Sitecore will resolve these issues, allowing for geographic redundancy of databases.

 Deployment

One of the joys of working with Azure Web Apps is the infrastructure available to automate deployments to production.  Web Apps leverage a sequence of deployment slots that allow for hot swapping of application code to roll deployments into production.

In order to assist in the speed of these deploys, Sitecore is providing ARM templates (recipes that allow for the quick creation of entire Azure architecture) and pre-built Sitecore configurations for typically complicated Sitecore role configuration, such as CM and CD.

What about MongoDB?

One thing not yet addressed is the requirement for MongoDB in order to utilize the xDB. There is always the Sitecore xDB Cloud offering, which externalizes the complexity of xDB, but for those who wish to host their data themselves, MongoDB can be complicated to host inside of Azure, as no managed solutions exist.   DocumentDB is a noSQL database that even has the ability to speak the MongoDB API.  It should be possible to support this alternative noSQL database in the future, allowing application hosting entirely inside of Azure.  In the meantime, ObjectRocket and mLabs both have offerings that help resolve these issues.

By the next iteration of Sitecore, we should be able to fully leverage Microsoft Azure to create a dynamic hosting architecture that saves money, reduces management effort and scales to fit the needs of your application.

Session: Sharing is Caring: Sitecore Customers talk about Personalization (Part 1)

In the business track this year at Sitecore Symposium, one of the best sessions was focused on specific clients and how they are leveraging personalization to significantly impact the experience for customers.  While many talks typically deal with hypotheticals that “could be done” this talk really focused on implementations that have actually proven results.

Kindercare

Kindercare is a child care and education provider with over 1700 locations, leveraging Sitecore to engage with customers and drive new business.  In implementing personalization for their site, the focus was on developing pattern cards, which address the core types of users they have on the site, such as “Development focused” vs “Value Conscious”.   For each of their five patterns, they scored that user on focus, the types of information they have available.   A “Development focused” user may be more interested in academics, expertise and milestone related content, while a “Value Conscious” user may be more interested in financial considerations and safety.

So far, that lines up with most usages of Sitecore’s persona based personalization I have seen, including the ones that we have implemented for our clients.  A set of user types are identified, typically referred to as a “persona” or “profile” and content is scored against those personas.   When users consume content that is tagged in a certain way, they begin to accumulate a score that associates them with a likely persona.  We can then use the rules engine in Sitecore to deliver content to that user that more closely aligns with that user.

The innovation that Kindercare had in this process was to run a survey of their users to validate the  assumptions.  They prompted users with a survey that asked them about the types of things that matter to them in their evaluation of a child care provider.  They were then able to compare the survey results with the hypothesis they had created in their digital marketing persona.  Rather than trust their own assumptions of what users interested in a certain type of content might also be interested in, they validated and tested those assumptions, enriching their personas.

Take away: Don’t stop at developing personas and scoring  your content against profile characteristics.  Test those assumptions and learn from your analytics what those users really want and update your model to reflect it.

Comcast

Comcast is a large cable television, phone and internet provider across the US.  Their home offerings are marketed under the xfinity brand, which is delivered with Sitecore.  For Comcast, they operate in a wide range of markets and the primary way they want to segment and personalize content to their users is by where they live.

While Sitecore’s IP Geolocation service will potentially return the country, state, city and other information about an IP address, the technology is limited in accuracy because IP addresses can be provisioned broadly across the country.   The information tends to be more accurate in metro areas than in rural ones.  Comcast is in a unique situation to have better information about the location of an IP address, as it is the internet provider for its customers.

Delivering relevant packages and messaging to the local market helped Comcast see a 30% uplift on sales, predominantly by delivering more targeted offers on their homepage for the local geography.

In addition to the focus on geography, Comcast is also customizing the offers they put forward based on other information they have about the user.  Comcast is leveraging Sitecore Federated Experience Manager (FXM) to allow for a more customized offer, even as the user is still anonymous.  The primary use case of the FXM is to monitor traffic on non-Sitecore websites using Sitecore’s xDB and deliver personalized Sitecore content to those sites using JavaScript.  Comcast has turned this process upside down, delivering a standard site experience via Sitecore, but leveraging FXM to change that content via JavaScript once the page has loaded, as more information about the user is made available.

The client-side JavaScript has access to more information about the use through third party cookies and call outs to other services to gather data.  Information, such as gender, interests and previous purchase history can be gathered about the user.  Utilizing the FXM, personalized content is then requested from Sitecore to ensure that the user’s first visit on the page gives them an offer that is the right message at the right time.  Imagine that I’m a person who loves sports and going to the Comcast homepage automatically brings up the NFL Sunday ticket package as the primary offer.  What kind of impact would that have on my likelihood of purchasing cable.

Take away: Federated Experience Manager (FXM) can allow you to use third party cookie data available on the client-side to personalize the experience based on actions that have not taken place on our site.

I loved the real world lessons on personalization that were brought by these Sitecore clients.  I was unfortunately unable to attend the second session, which focused on Cirque de Soleil and how they are using personalization.  If you were there, let me know what you thought.

Sitecore 8.2 and the move to Azure

I was excited to see the announcement that Sitecore 8.2 has been released.  As a Sitecore MVP, I have had some early access to the platform and there are lots of great things to talk about that will warrant their own blog posts.  One thing that has not received as much press is a small indication that  Sitecore is more closely aligning with Microsoft Azure.

Sitecore 8.2 is the first version that officially supports using Microsoft Azure SQL, which is the Platform as a Service (PaaS) offering of SQL Server provided in Azure.  Up until this point, the only supported way that you could run Sitecore in Azure was either fully Infrastructure as a Service (IaaS) or through the Sitecore Azure module, which does provide full support for many of Sitecore’s modules.

Sitecore now ship with two options for restoring the standard databases that Sitecore provides.  The .mdf and .ldf files common to any on-premise implementation are still there, but there is  now a folder of DACPAC files utilized to deploy to Microsoft Azure SQL.  These files can be deployed to your instance in Azure either through Powershell or through the standard SQL Server Management Studio 2016.  More information on restoring these here.

Microsoft Azure SQL allows teams to focus on the application level architecture, relying on Microsoft Azure to handle scaling, backups and performance of SQL Server.