New Features in SharePoint 2013 – Part 5 (Themes)


Themes

SharePoint 2013 has a new theming engine and better graphical user modification options. This theming engine has been completely re-worked from bottom to top. Everything is now based on HTML instead of proprietary format, which includes support for newer standards like HTML5 and CSS3. It also means that PowerPoint is no longer used to create custom themes for site collections.

However we will get much richer themes and common building blocks for customizing them, so for example we will have

  1. Theme Gallery: You will get a new theme gallery in SharePoint 2013, below is an example of different theme templates that come out of the box. If you click either one of those, you will be able to customize that theme with different mix and max below elements to make it look exactly like you want to.

  2. A background image: Typically we will start by selecting a background image for the theme. If we have Office 2013 client, then we can typically drag and drop the image from the local file system on to the page and have that image automatically uploaded and displayed as a background image. If we don’t have Office 2013 client, then we can change the back ground image by simply uploading the image and apply the new background image to the theme. We can also remove a background image from the existing theme.
  3. A colors palette: Below the background image, we will have a color wheel. The color wheel contains a predefined set of color combinations and those colors will be used in the themes in our site. We can choose a color schema from a list of available color schemas to be applied across your sites
  4. Layout Dropdown: Below the color wheel we can find a layout dropdown. We can click on Layout dropdown that comes in 2 different site layouts, the 1st is V15 (which includes a quick launch bar) and 2nd is Belltown (which doesn’t include a quick launch bar). We can also create multiple site layouts programmatically and make them available for use.

  5. Fonts: Below layout dropdown is a Fonts section. We can choose which fonts to use within the theme.

  6. Live Preview Frame: Once you have gone ahead and selected your choices for the above areas, live preview pane automatically updates to display a sample image for the chosen combination.

  7. Try it Out: After we are satisfied with the way live preview pane looks, we can go ahead and click “Try it out” link to see changes being applied to our site. We will see our site and our content with the new theme. If we like it we can select “Yes, keep it”, otherwise we can select “No, not quite there” to get back to the theme selection page and refine it further.

Posted in SharePoint 2013 | Leave a comment

New Features in SharePoint 2013 – Part 4 (Request Management Functionality)


SharePoint Server 2013 Preview provides unparalleled levels of availability and site resilience, while also reducing the cost and complexity of deployment to environments of diverse sizes. Building on the native capabilities introduced in SharePoint Server 2010, improvements in SharePoint Server 2013 Preview provide a simplified, unified solution for high availability, disaster recovery, and backup.

Because of the length of this article, I am dividing this article into 2 blog posts. The 1st section deals with Request Management functionality, the 2nd section deals with Request Management Configuration.

Request Management Functionality:

Request Management in SharePoint Server 2013 Preview enables IT to prioritize and route incoming requests through a rules engine that applies logic to determine the nature of the request and the appropriate response.

Request Management can be used to:

  • Route requests to servers with good health characteristics based on a new weighting schema.
  • Prioritize requests by throttling lower priority requests to preserve resources for those of higher priority.
  • RM can prioritize requests by throttling lower-priority ones (bots) to serve higher-priority ones (end-users).
  • Route specific request types to other servers, either within or outside of the farm handling the request.
  • RM can send all requests of specific type, like search for example, to specific machines.
  • RM can route to WFEs with better health, keeping low-health WFEs alive.
  • RM can send heavy requests to more powerful WFEs.
  • RM can identify harmful requests and deny them immediately.
  • Identify and block known bad requests such as web robot.
  • Isolated traffic can help troubleshoot errors on one machine.

This is a really exciting functionality as it will help us in large deployments by routing appropriate traffic to the server. The purpose of the Request Management feature is to give SharePoint knowledge of and more control over incoming requests. Having knowledge over the nature of incoming requests – for example, the user agent, requested URL, or source IP – allows SharePoint to customize the response to each request. RM is applied per web application, just like throttling is done in SharePoint 2010.

How does Request Management work (Functionality)?

Request Manager is comprised of 3 primary components and it follows them step by step to get to the WFE (Web Front End).

  1. Request Throttling and Routing Component
    1. Request manager check for throttling rules for every request and if a matching rule is found, then the request is throttled. If the Throttling Rule is not found, then the request manager looks for routing rules.
    2. If a routing rule is identified then the request managers, looks for a list of WFEs to send the request to.
  2. Request Prioritization
    1. Request manager filters WFEs to only ones healthy enough for the request
  3. Request Load Balancing
    1. Request manger will then load balance the WFE’s and select a single WFE to route to, based on weighting schemes like health

How are Routing Rules Identified?

Routing Rules are defined for every request and Routing rules are associated with Machine Pools. Machine Pools contain servers (Single or Multiple) and servers use weights (static, health) for routing. So, whenever there is a new request, it will be mapped against a routing rule and then will be sent to appropriate machine pool, which will be again sent to a server depending upon the weights.

Weights can be divided into 2 categories

  1. Static weights are constant for WFEs (these are default values for all the WFE’s and Admins can change those static weights)
  2. Health weights change dynamically based on health scores (a server with less health score is less likely that a request will be routed to it)


RM Routing Rules and Execution Groups:

To summarize, routing to a server in a Machine Pool is based on matching a “Routing Rule”. Just like the way servers are present in a Machine Pool, Routing Rules are placed in Execution Groups (numbered 0-Default, 1, 2). Rules are evaluated in each Execution Group (Execution Groups are executed in order form 0 to 2). As soon as a match is found no more Execution Groups are evaluated. All machines from pools that match any routing rules are grouped together to determine possible target servers. This means that you create your most important rules in Execution Group 0 (default).


RM Routing Rules Continued:

There are some important caveats to remember about routing rules

  • If no rules are matched, then the request will be sent to any server that is NOT in any machine pool for any rule (THIS MAY CHANGE AFTER RTM)
  • In a one server farm that means nothing will route if no rules match, so the alternative is to create a “catch all” rule that matches everything. Just put it in Execution Group 1 or 2 so it’s the last match

RM – Why not throttling?

SharePoint 2010 has throttling but there is room for improvement. It uses a health score system in which WFEs attach their health info to all responses that go back to clients

The drawbacks from this approach were:

  • It was the clients’ responsibility to honor the health scores, so clients would get health score and it was up to them to honor them or not.
  • It did not preclude WFE failure; it did no good because Client doesn’t have any way of knowing about it. It could get a server response back, but it doesn’t know that subsequently went down.
  • Clients could be shown server busy messages from a poor-health WFE when other better-health WFEs were available to service those requests

RM – Throttling Rules:

Request Management builds on the throttling features of SharePoint 2010 and implements it on server side using rules. Routing rules process requests; throttling rules stop requests. It’s much like throttling in SharePoint 2010, only more sophisticated, you create criteria for the throttling rule, and if the criterion is met the request is throttled (discarded). The process and PowerShell for creating throttling rules is very similar to routing rules

What Criteria can you use for Throttling and Routing?

When creating a routing rule or throttling rules, Rules can match on these properties:

  • What’s the URL that has been requested?
  • Who is the UrlReferrer?
  • What’s the UserAgent?
  • What’s the Host that has been requested?
  • What’s IP address that request was submitted from?
  • What HttpMethod is requested?
  • What SoapAction is requested?
  • You can also add a CustomHeader that request management will evaluate?

You can evaluate values using these methods to see if a rule matches?

  • StartsWith
  • EndsWith
  • Equals
  • Custom RegEx

RM – Heavy Client Application:

You have a heavy load on the system with many browser requests. Notebook sync requests start coming in from OneNote, the OneNote requests start adversely affecting the browser requests so you can add a throttling rule is added to deny OneNote requests: Here is an example where you can create a throttling rule and the criteria would be based on UserAgent Property and we are going evaluate the property is by using a Regex to check for OneNote.

Rule: Deny requests with UserAgent of regex = “.*Microsoft Office OneNote 2010*”

Based on this rule RM denies OneNote requests. When system load dies down, the admin can remove the throttling rule

Other Options:

  • Rule could use an expiration to automatically deactivate the rule at a certain time. (Example: A throttle rule can be activated at a certain time and deactivated at a certain time)
  • Rule could use a health score threshold to activate. (Example: A throttle rule can be activated when my health score gets above 7 or 8, where the health has adversely impacted)

RM – Routing Weights:

RM uses static weights and health weights to help determine which servers are going to get the request.

  • Static weights are associated with WFEs so certain ones will always be favored when selecting. This gives added weight to more powerful WFEs and less to weaker machines. (Example: When we add hardware to the farm, we can increase the static weight to be higher than existing servers in the farm)
  • Health weights are used to even out load and keep “sick” WFEs going. So health scores run from 0 to 10 where 0 is the healthiest and therefore will get the most requests; this score is used to derive the health weight. So WFEs start with a healthy weight; the Policy Engine health rule updates health weights dynamically – you cannot change it manually

RM Scenario – Health Based Routing:

A series of requests come in; one WFE is in poor health, while two others are in good health.

RM evaluates the following:

Health information: { [WFE1, sick], [WFE2, healthy], [WFE3, healthy] }

Based on this RM routes most of the requests among WFE2 and WFE3. It is still random routing, but greater weight is given to healthier machines. Alternatively the admin could remove WFE1 from the routing pool, allow it to complete its requests then return it back to the pool

Please check 2nd section for Request Management Configuration in SharePoint 2013 – Part 4.

Posted in SharePoint 2013 | Leave a comment

New Features in SharePoint 2013 – Part 3 (Cache Service)


Distributed Cache

Data-driven applications have become increasingly prevalent as data is consumed from more diverse sources, such as business applications, syndicated feeds, and social contexts. SharePoint Server 2013 Preview includes a new Distributed Cache Service built on the reliability of Windows Server AppFabric Distributed Caching. Distributed caching helps to ensure that no request takes too long.

So, coming to our questions how is this Distribution Cache setup?, What is the logic behind it? To understand it, we shall get deeper into it in the below topics.

  • Windows Server AppFabric extends Windows Server to provide enhanced hosting, management, and caching capabilities for Web applications and middle-tier services. The AppFabric hosting features add service management extensions to Internet Information Services (IIS), Windows Process Activation Service (WAS), and the .NET Framework 4. This includes Hosting Services and Hosting Administration tools that make it easier to deploy, configure, and manage Windows Communication Foundation (WCF) and Windows Workflow Foundation (WF) based services. The AppFabric caching features add a distributed, in-memory object cache to Windows Server that makes it easier to scale out high-performance .NET applications, especially ASP.NET applications.

  • How do you start Distributed Cache?

    The distributed cache feature is enabled by default and the distributed cache service is automatically started on all web and application servers in a farm. A new Windows service – the AppFabric Caching Service – is installed on each server in the farm when SharePoint is installed. It is managed via the Services on Server page in central admin as the “Distributed Cache” service and SharePoint_Config DB keeps track of which machines in the farm are running the cache service. In very large environments distributed cache can be offloaded to dedicated server.

  • How does it improve performance?

    Distributed Caching improves performance by

    • Caching My Site social data, such as news feeds,
    • Caching authentication tokens,
    • Workflows

       

  • Distributed Caching has its limitations
    • SharePoint 2013 uses caching features that cloud-based cache (Windows Azure Cache) does not support at this time, so only local cache hosts can be used; may change in the future
    • SharePoint ONLY supports the version of caching that it ships – you cannot independently upgrade it.

       

  • Service Account Requirements for Distributed Cache:
    • The farm account is used as service account for Cache Service
    • Like user profile service in SharePoint 2010, during setup the service account should have elevated privileges (i.e. local admin)
    • After setup is complete you should lower the privileges for the account

       

  • Cache Architecture:
    • For caching in farm, scale points have not been determined yet
    • How many servers are needed, what resources should be built out (CPU, memory, etc.)
    • More data will be available after Beta 2

       

  • Cache Physical Architecture:

     

  • Cache Server Performance:

    There are hundred(s) of performance counters that are exposed via developer’s dashboard. These counters can help in determining whether we need additional Cache Servers, CPU, or increase in the size of the memory, higher through put i/o, better disks, etc. Some of the performance counters that are important for Cache server performance are

    • Number of reads, number of writes, number of hits and number of misses
    • Time to read, time to write
    • Total I/O (how much data has been transferred in a given period of time)

       

  • Cache Service Health

    The following health rules have been created to help you track the Cache Service. Most of these health rules are present in Availability section.

    Availability:

    • One of the cache hosts in the cluster is down. We might have to restart the service on this box
    • Cache host is in throttled state. Which means we have availability issues, we may need to restart it
    • The high availability node for SharePoint distributed cache is not available – happens when there are less than 2 servers running the cache service

       

    Configuration:

    • Firewall client settings on the cache host are incorrect. Which means the Cache Host can’t be contacted
    • There exists at least one cache host in the cluster, which SP doesn’t know about – happens when the cache service is disabled in SharePoint but AppFabric Caching Service is running on the machine
    • Cached objects have been evicted – indicates eviction happened across the cache cluster. Not bad in and of itself but may be a clue if it happens frequently and/or there are performance issues

Come back soon to learn about New features in SharePoint 2013 – Part 4 (Request Management)

Posted in SharePoint 2013 | Leave a comment

New features in SharePoint 2013 – Part 2 (SQL improvements)


SQL Improvements: Numerous SQL Server improvements to make SharePoint faster and also SharePoint will now support SQL Server 2012 from back end.

Data Platform

Scale and performance improvements are evident across SharePoint Server 2013 Preview, with depth that extends to the database layer. Database improvements in SharePoint Server 2013 Preview take advantage of enhancements and capabilities through SQL Server 2008 R2 SP1, including:

  • Conforming to Microsoft SQL Azure™ compliance criteria.
  • Removing redundant and unused tables and indices to track links
  • Reducing I/O operations while browsing document libraries.
  • Using SQL sparse columns to simplify SharePoint database schema and optimize data access.
  • Improving large list dependencies.

SQL Server 2012

SQL Server 2012 contains a new feature called AlwaysOn Availability, which is a high-availability, disaster recovery solution that can be used when using SharePoint 2013 with SQL Server 2012. In addition to the AlwaysOn Availability, SQL Server 2012 also has reduced input/output (I/O) when browsing libraries, which will improve performance in SharePoint 2013.

Databases

Seven SharePoint 2010 databases have been deprecated in SharePoint 2013, but there are five new databases added to SharePoint 2013. The databases that have been removed include four project related databases: Project Publish, Project Archive, Project Draft, and Project Reporting. These databases will be replaced by a single database called Project Service.

The Search Property Database has also been removed, but in its place you will find a Links Store Database associated with the Search Service Application. The Search Service Application Analytics Reporting Store Database stores the information that was formerly stored in the Web Analytics Report and Web Analytics Staging databases, meaning the functionality of the Web Analytics Service Application has been integrated into SharePoint Search.

The five new databases are:

  1. Project Service
  2. Search Service Application Links Store Database Library
  3. Search Service Application Analytics Reporting Store Database
  4. App Management Service Database
  5. Translation Service Database

(Thanks to Brian Alderman for the insight)

Come back soon to learn about New features in SharePoint 2013 – Part 3 (Cache Service)

 

Posted in Uncategorized | Leave a comment

New Features in SharePoint 2013 – Part 1 (Shredded Storage)


 

Architecture:

 

SharePoint 2013 is fundamentally built on the same architecture as SharePoint 2010. From architecture perspective SharePoint 2013 is similar to the previous SharePoint 2010 model, the model itself hasn’t changed that much, it uses similar service applications architecture and it how ever has lots of platforms improvements and capabilities like Shredded Storage, Distributed Cache Service, Request Management, SQL Improvements, Themes, Sharing, etc., I will try to add as much information as possible for each of these improvements.

Shredded Storage:

Shredded Storage minimizes the actual files sizes or change set sizes or what may store within the files. This means that SharePoint 2013 is sending that only relevant information from servers to the end user that he is modifying and in turn stores that same piece of information to the database and it doesn’t update the whole BLOLB as such, this will further improve the performance.

Concept:

To make it clearer, documents/attachments present in a library/list are stored as BLOBS (Binary List Objects) in the Content DB (SQL Server Database). So, if version history is enabled for a document library then it stores each version of the document as a new BLOB with in the SQL. This BLOB size includes the document and all the metadata associated with it. Every time there is a new version it creates a copy of the BLOB and stores it in the database.

Doing the math here, a 10MB document (with some metadata associated to it) in a document library with 10 versions enabled will need

10 (version) * 10 MB (Document Size + Meta data) = 100MB of BLOB size in SQL Server Content DB

This architecture introduces 2 problems into the way SharePoint operates

Problem 1: SQL server has lots of redundant BLOB storage making it difficult to manage

Problem 2: Performance has degraded by serving all the unnecessary data to the end user

Microsoft solved both of these problems by using “Shredded Storage in SharePoint 2013”. Shredded storage solves both of these problems by introducing the concept of Shredding. At the highest level, SharePoint 2013 chunks or pages the BLOB into numerous smaller shreds. So a single BLOB is now made of multiple shreds. The advantage of this architecture is that only the “shreds” that have been modified will be copied and saved to the SQL Server. For example, if versioning is enabled and a user makes a change to a document, only “changed shreds” are added to the storage foot print of that document. Shreds that have not changed are simply associated with both versions.

This will help in significantly improving the storage utilization, that same 10MB file with 10 versions may be consuming 22MB instead of 100MB.

Shredded storage also reduces the amount of information that a file has to be retrieved by the Web server from the SQL Server, so Input/ Output operation improves significantly.

The good news is this architecture is already built into the SharePoint 2013 system, so we don’t need to start any services on the server nor modify the web.config. On the other hand Web Server and SQL Server have additional workload that helps in improving the performance and storage.

I hope this clarifies some confusion around Shredded Storage in SharePoint 2013

(Thanks to Dan Holme for wonderful explanation)

 

Come back soon to learn about New features in SharePoint 2013 – Part 2 (SQL Improvements)

Posted in SharePoint 2013, Shredded Storage | Leave a comment