From time to time there are questions around using Orchestrator as a batch robot, for example run a set of file copy jobs every night. In general Orchestrator was not designed as a job scheduler, even it if would work in small scale. Challenges you can run into when trying to this are

• There is not a single view to show when you have schedule which runbooks. For example if you want to check all runbooks configured to run every night you have to browse through all your runbooks in Runbook Designer.
• There is not a single view to show all the calendars you have configured and how they are configured.
• There is not a single view to see if all runbooks was successfully last ran, for example if you have a number of runbooks running every night you want to have one view in the morning to verify that everything worked.
• There is no easy way to import a calendar from another tool
• By default a runbook server can run 50 jobs at the same time. You can reconfigure this (http://technet.microsoft.com/en-us/library/hh420378.aspx) but you should consider the resource requirements of the runbooks on a particular server. Lets say you start one runbook that checks for new files in a folder and then for each new file you invoke another runbook. What if there are 1500 new files in that folder? 1500 running runbooks would require a lot of runbook servers to run all that at the same time. If you don’t have another runbook servers a queue would be built up, which could result in long delays and affect other runbooks.

If you still want to schedule a runbook, for example to run every 12 hours, you can use a Monitor Date/Time activity. A disadvantage is that the runbook will be running all the time, monitoring the schedule. As the runbook is running all the time, it will use one of the 50 (by default) slots on your runbook server.

If you want to run your runbook 01.00 on Sundays only you can also use the Monitor Date/Time activity, but you need to combine it with a runbook level schedule. Create a new schedule, check only Sundays, and configure your runbook to use the new schedule. You configure the runbook to use the schedule by right-click on the runbook tab, select properties and then click schedule.

After you have configured a runbook to run for example every Sunday at 1 am you start it. The runbook will get the green “running” icon and in the database it will be marked as pending. Once it gets Sunday at 1 am the runbook will run. After the schedule time window the runbook will stay in running status, until next schedule time window, even if it is not allowed to execute between the schedule time windows.  If the runbook tries to run other times a platform event will be generated.

Instead of using Orchestrator to trigger runbooks based on a schedule you can use Windows Schedule Tasks, see this blog post from the Orchestrator Engineering team. I have two related blog posts around this, one about execute runbook later and one about runbook servers in different time zones.

Note that this is provided “AS-IS” with no warranties at all. This is not a production ready management pack or solution, just an idea and an example

Orchestrator support a number of data manipulation functions. These functions can be used to manipulate string and  convert it into a usable form. You can also perform simple arithmetic operations, such as calculating sums and differences, and performing division and multiplication operations. For a complete list of the functions that you can perform see Technet. One nice thing about these data manipulation functions is that you can combine them.

Example, a simple text file

I use a Read Line activity to read the line, and I can then manipulate the data, in this figure I do that in a Send Platform Event activity

The result will be

John Connor,Skynet,555-123456

John Connor

John

Jo

JO

555-123456

555

123456

124011

First line is the text line as in the text file. Then it is
String divided on “,” and showing first part
String divided on “,” and showing first part, then dividing that answer on ” ” and showing first part
String divided on “,” and showing first part, then dividing that answer on ” ” and showing first part, then showing two first letters from the left side
String divided on “,” and showing first part, then dividing that answer on ” ” and showing first part, then showing two first letters from the left side, then UPPER means converting that string to uppercase

About the numbers

String divided on “,” and showing third part
String divided on “,” and showing third part, then dividing that answer on “-” and showing first part
String divided on “,” and showing third part, then dividing that answer on “-” and showing second part
String divided on “,” and showing third part, then dividing that answer on “-” and showing first and second parts, then sum the two numbers

I have updated the “runbook validator” again. Added a couple of new checks and also added a table of contents in the beginning of the report. You can download the new version here, 20130419_SCO_CHECK.

Since I started to work with Opalis I have been surprised to see how fast people build their first runbook (policy back in Opalis). When showing Orchestrator for a group of engineers it doesn’t take many minutes before they have a running workflow. A disadvantage when it is too easy, drag-drop-done, is that Orchestrator is some times not used as it should be used. In Orchestrator the challenge is that many engineers quickly drag and drop a couple of activities to the runbook, click Start and then lean backward. Most likely they have a working runbook, but are they really working? Is it built as it should be? Is it tested? I have written a couple of blog posts around best practices and the areas that the “runbook validator” checks

• Fault tolerance in Opalis polices,
• Building a log for your runbooks
• Check in history
• Should this runbook be running
• Job concurrency in Orchestrator
• Consider Invoke Runbook activities when doing a update and dependencies between runbooks
• Activities not using default account and run a runbook with a specific account
• Link delays

The “runbook validator” is a set of runbooks that will check all other runbooks according to a number of best practices. A report file will be generated with the result. All runbooks use a number of variables, like Orchestrator database server name and database name. Don’t forget to update them before you run the check.

If you have any ideas what I should add, please post them as comment to this post or send me an e-mail. Please note that this is provided “as is” with no warranties at all. Also note this is all based on my ideas and is not a “Health check” or Microsoft official guidelines.

Runbooks should be administered using version control. For each version deployed into production, create a package including the runbook and documentation. The documentation should include an explanation of the runbook, including all parameters, global settings and variables. As Orchestrator don’t include version handling it is a good idea to keep earlier runbook versions as export files, making sure you can rollback to earlier versions easy. You should also have a change management process around releasing new runbooks to production to make sure they are tested and signed off correct by the different teams.

As an additional layer of your change process, or maybe in lack of a change process, you can use check in comments in Orchestrator. In the Runbook Designer console you can enable “Prompt for comment on check in”.   This is not enabled by default but it is something you should enable at least in your production environment. The result is that each time you click check in in the Runbook Designer console you need to input a comment. It is very easy way to keep track of changes.

You enable check in comments from the Options menu > Configure, in the Runbook Designer console.

All check in comments are written to the Orchestrator database, in the CHECK_IN_HISTORY table. Together with the POLICIES table and the OBJECTS_AUDIT table we can list all comments with account name, runbook name and date. We can also use SQL Server Report Builder to build a report to show this data. The following SQL query will show all check in information.

SELECT CHECK_IN_HISTORY.DateTime AS [Check In Time], POLICIES.Name AS [Runbook Name], CHECK_IN_HISTORY.Comment AS [Check In Comment], SIDS.Account, POLICIES.Version AS [Current Version in database]FROM CHECK_IN_HISTORY INNER JOIN POLICIES ON CHECK_IN_HISTORY.ObjectID = POLICIES.UniqueID INNER JOIN SIDS ON CHECK_IN_HISTORY.CheckInUser = SIDS.SIDWHERE (POLICIES.Deleted = 0) ORDER BY [Check In Time] DESC

“Current Version in database” is a value we get from the POLICIES table. Each time we click check in Orchestrator will add one to that value, so if we check out and check in a runbook 20 times it will show version 20. As we get this value from the current checked in runbook it will show the same “Current Version in database” for all check in for the same runbook, so it is not the version that was checked in at that time.

You can download my example report here, CheckInHistory. Note that this is provided “AS-IS” with no warranties at all

I in this blog post I would like to share an idea around measure how much time you are saving by using Orchestrator. Many organizations  spend weeks building cool runbooks, but are they really saving hours, or did the building phase took more hours then they will ever save? My idea is to build a report that show number of hours saved and compare it with number of hours building the runbook. To do a report on this we need some info about each runbook

• How often is the runbook running?
• How long would it take to do the same task manually?
• How much would manually hours cost us?
• How much did it cost to build the runbook?

There is another important aspect on this. Runbooks will always be executed the same way. When someone is doing a task manually there is always a risk that the do something wrong and we need to spend hours/day fixing that. That “risk” is difficult to estimate. If a solution includes multiple runbooks it could be difficult to estimate each runbook, then it might be configure hours on the “parent/root” runbook and not all the child/functions” runbooks.

All this info can be stored in the Service Manager database. I extended the runbook class with extra attributes.

• Runbook owner: Engineer responsible for the runbook
• Runbook Category: a category from a list item, a way to group runbooks
• Manual hours: Number of hours it will take to perform the task manual
• Manual cost: The cost per hour when performing the task manual
• Build cost: The cost to build the runbook

We can query the Orchestrator database to get runbook instance history, from the Microsoft.SystemCenter.Orchestrator.Runtime.RunbookInstances view. We can query the Service Manager database to can get runbook cost and hours, from the MTV_Microsoft$SystemCenter$Orchestrator$RunbookItem table. But out of the box Orchestrator only store 500 entries of log data, we should also try to keep the Orchestrator database small, so it is not a really good data source for this report. Instead I have created a new database where I write a daily summary for each runbook.

I have a runbook that read from the Orchestrator database and writes to the Log database. This runbook gets all runbook objects from Service Manager that have a “Build Cost” specified. Then it query the Orchestrator database for success and failed jobs for previous day,  and writes that to the Log database. It also writes a summary of number of hours saved (SM Manual Hours x Number of success jobs) and total amount of money saved (SM Manual Hours x Number of success jobs x manual hours cost).

We can then use SQL Report Builder to build a report to show the data. In my example I have a table and a smiley face icon. The smiley face icon show if I have saved more money then spent on building runbooks. I can also drill down into each runbook category and see data about each runbook. The report includes two parameters, start and end date, to report time frame.

You can download my example files here, OrchestratorScoreCard. It includes the Service Manager extension, the runbook and the SQL report file. Note that this is provided “AS-IS” with no warranties at all. This is not a production ready management pack or solution, just a idea and an example. As always, thanks to Patrik for support and good discussions around System Center.

Last week I was working on private cloud solution that included around 50 runbooks. In the end of the development phase we needed to make sure all runbooks was following best practices, including naming convention and design standard. For example all links between activities that had a name including “IF” should be updated to orange color, and all runbooks named “Link” should be updated to green color. As we had a lot of runbooks I was thinking about a quick way to updated them all, instead of manually updating each link. We also wanted to update some activity names based on the configuration.

The answer was a couple of SQL queries

To update all non-deleted (Deleted = 0) runbooks that have black (default) color and named Link (default) to green link color

Update LINKS SET Color=’65280′ WHERE Color = ‘0’ AND UniqueID IN (select uniqueID from OBJECTS WHERE ObjectType = ‘7A65BD17-9532-4D07-A6DA-E0F89FA0203E’ AND Name = ‘Link’ AND Deleted = ‘0’)

As you can see we set the color to 65280,

• Green is 65280
• Black is 0
• Red is 255
• Orange is 4227327

To updated all non-deleted links that have name that contains “warning” to red link color, run

Update LINKS SET Color=’255′ WHERE Color = ‘0’ AND UniqueID IN (select uniqueID from OBJECTS WHERE ObjectType = ‘7A65BD17-9532-4D07-A6DA-E0F89FA0203E’ AND Name like ‘%Warning%’ AND Deleted = ‘0’)

If you want to update link width for all links named “Link” you can run the following query

Update LINKS SET Width=’3′ WHERE UniqueID IN (select uniqueID from OBJECTS WHERE ObjectType = ‘7A65BD17-9532-4D07-A6DA-E0F89FA0203E’ AND Name = ‘Link’ AND Deleted = ‘0’)

If you want to update all Invoke Runbook activities with a name starting with “Invoke”, and set the name to the runbook it invoke you can run the following query

UPDATE OBJECTS
SET objects.Name = TRIGGER_POLICY.PolicyPath
FROM OBJECTS, TRIGGER_POLICY
WHERE objects.UniqueID = TRIGGER_POLICY.UniqueID
AND objects.name LIKE ‘INVOKE%’ AND Deleted = ‘0’
GO

You don’t need to checkout your runbook to  apply the modification, just refresh the console after the update.

Please note that is unsupported as we modify settings in the database direct. This is provided “AS-IS” with no warranties at all. This is not a solution for your production environment, just a idea and an example.