Home » Orchestrator (Page 2)

Category Archives: Orchestrator

Contoso.se

Welcome to contoso.se! My name is Anders Bengtsson and this is my blog about Azure infrastructure and system management. I am a senior engineer in the FastTrack for Azure team, part of Azure Engineering, at Microsoft.  Contoso.se has two main purposes, first as a platform to share information with the community and the second as a notebook for myself.

Everything you read here is my own personal opinion and any code is provided "AS-IS" with no warranties.

Anders Bengtsson

MVP
MVP awarded 2007,2008,2009,2010

My Books
Service Manager Unleashed
Service Manager Unleashed
Orchestrator Unleashed
Orchestrator 2012 Unleashed
OMS
Inside the Microsoft Operations Management Suite

Export runbooks with SMA

This runbook can be used to export SMA runbooks based on runbook tag. All runbooks with the tag you specify as a parameter will be exported to the C:\TEMP folder. Remember to update $WebServiceEndPoint to your SMA web service.


workflow Export-Runbook
{
param
(
[Parameter(Mandatory=$true)]
[string] $Tag
)
$WebServiceEndpoint = "https://wap01"
InLineScript {
$runbooks = Get-SMArunbook -WebServiceEndpoint $using:WebServiceEndpoint | Where-Object {$_.Tags -like "*$using:Tag*"}
foreach ($i in $runbooks)
{
$runbooktoexport = Get-SmaRunbookDefinition -Type Draft -WebServiceEndpoint $using:WebServiceEndpoint -name $i.RunbookName
$outpath = "C:\temp\" + $i.RunbookName + ".txt"
$runbooktoexport.Content | Out-File $outpath
}
}
}

Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.

List with Choice columns and the SharePoint IP

I have read in a number of forums posts that there are challenges using SharePoint list with columns of type choice together with the SharePoint integration pack (IP). The challenge is that if you use the Get List Item activity together with a choice column you will notice the result is blank/null, no data is returned by the activity. Other column types, for example Single line of text, works fine. Choice columns is columns with a menu to choose from, for example drop-down menu, radio buttons or checkboxes. This blog post will show how to get data from Choice columns with the SharePoint web service.

In this example I have a list named Virtual Machines. I use this list to manage my virtual machines in my sandbox, for example if I create a new item Orchestrator will notice that and create a new virtual machine based on the list item. This list includes two columns of type Choice, one for virtual machine size and one for virtual machine location.

 

Figure 1 SharePoint list settings and SharePoint form

 

Figure 2 show the runbook monitoring the Virtual Machine list in SharePoint for new items. When a new item is found the runbook use the Ask web service (Invoke REST Service) activity to query SharePoint for all information about the new list item. Most of the data the runbook gets from the Monitor activity but not data input in a Choice column, in this example the Monitor activity publish Size and Location as blank/null values. Figure 3 show the configuration of the Ask web service activity. The Title value in the URL is the server name inputted in SharePoint as a single line of text, which is picked up fine by the Monitor activity. The Ask web service activity will return all data for the specified server, including data inputted in a Choice column, the Query XML activity filter out and return only the Location attribute. Figure 4 show the configuration of the Query XML activity. Once the runbook have the location it will either invoke Virtual Machine Manager (VMM) to build the virtual machine or Service Management Automation (SMA). I use the SMA feature of Orchestrator 2012 R2 for cross cloud management, building virtual machines in Azure. I could build the Azure virtual machine from a Orchestrator runbook too, but it is handy to use PowerShell workflows in SMA for that. I use VMM for building virtual machines in my private cloud.

 

Figure 2 Runbook monitoring SharePoint list

Figure 3 Query web service settings

Figure 4 Query XML to get server location

Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.

NOTE: THERE IS A UPDATE VERSION OF THIS IP. THAT WORKS WITH LIST ITEMS, DOWNLOAD

Temporary Permissions with System Center

Do you have multiple accounts for administrator of your IT environment? Do you have for example one local administrator account, one Active Directory administrator account, one account to administrate some business service and one account that you use to log on to your workstation and read your e-mail? Do you remember all the passwords? I often see customers having a challenge remember all passwords and when to use which account.

Another funny (or scaring) side of it is that everyone is more or less administrator of everything, it is just a question about clicks. For example an engineer is not a domain admin but administrator of Orchestrators that integrates with Active Directory using a domain administrator account. Another example could be an engineer without any administrator accounts except is Operations Manager administrator, with indirect access to all servers as local system, including domain controllers.

One thing that also crossed my mind is why do so many engineers have permissions to do 200 tasks, when you normally only do 20 tasks? Why do you have an account that could delete the entire domain when all you do is reset passwords and add members to groups? There is an imminent risk of human errors.

I am no security expert but I want to show a way to minimize number of administrator accounts floating around, and maybe minimize human errors. The scenario is that no engineers have more than one account. That account is a normal user account, no special administrator permissions at all. When the engineer are about to do something special he or she request permissions and also describes the work that will be done. The engineer also select number of hours permissions are needed.

Figure 1 The Service Request form

The service request templates includes the following activities

  1. Runbook, update approval step. This step will look which system the requester has requested permissions to, and then depending on the system configure the manual approval step. If it is for example Active Directory the runbook activity will add a group of people that can approve Active Directory permissions.
  2. Review Activity, “managers” or “owners” approve temporary permission
  3. Runbook, grant temporary permissions. A runbook grants the requester permission to the system

Figure 2 Service Request Template

 

The first runbook, 2.7.2 in this scenario, is the runbook that will update the review activity based on the request. The review activity is configured in the service request template without any reviewers, it is the 2.7.2 runbook that will populate the list of Reviewers. A dynamic approval step J

The second runbook, 2.7.1 in this scenario, is the runbook that will grant permission to the requester. It gets the user requesting permissions and add that user to a group in Active Directory. The Map System To Group activity will decide which security group to add the user to, depending on system. In the end of the runbook it updates the service request with information that permissions has been granted and when the permissions will be removed. The Format Timestamp activity is used to get expire time for the permissions. But there is one more extra important activity in the runbook. The Write to Temp Permissions database activity. For each permission granted this activity writes a row to a SQL database.

 

Figure 3 Temp Permissions database

The last runbook, 2.7.3 in this scenario, is removing permissions. Every 10 minutes it query the Temp Permissions database for permissions to remove. If there is any permissions to remove, based on the ExpireDate column, the runbook will remove the user from the security group and remove the record in the database. There will still be logs in the technical log database for Orchestrator, more info about logs here. You will also have a work item, a Service Request, in Service Manager which information about the temporary permission, including requester, approver, description and system.

 

You can download the runbooks, 20131204_TempPermissions. Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.

Q&A around Orchestrator Integration Packs when upgrading

Last weeks I have received a lot of questions around what will happen to runbooks and integration packs when upgrading to Orchestrator R2. Here are top five questions and answers.

Can I integrate with System Center 2012 SP1 or 2012 from Orchestrator 2012 R2?
Yes, R2 supports running Integration Packs from both 2012 SP1 and 2012

Can I integrate with both System Center 2012 SP1 and 2012 R2 from same Orchestrator?
Most likely the R2 integration pack will work for both. But from a support point of view you need to have the 2012 SP1 integration pack AND the 2012 R2 integration pack. The challenge is that the R2 integration pack will replace/upgrade the SP1 integration pack. To be fully supported you will need two Orchestrator envrionemnts, one with 2012 SP1 integration pack and one with R2 integration pack.

I have a number of runbooks today working with Service Manager 2012 SP1, I am planning to upgrade Orchestrator to R2, will these runbooks work?
Yes, you will keep the same integration packs in R2, in this scenario the SP1 integration pack.

I have a number of runbook today, do I need to rebuild them after R2 upgrade?
No, you don’t. The same integration packs and activities will be in Orchestrator after upgrade

I have System Center 2012 SP1 today, I am planning to upgrade both Service Manager and Orchestrator. Do I have to update all my Service Manager runbooks?
No, when you have upgraded Service Manager to R2 you will upgrade the integration pack for Service Manager to R2 too. Orchestrator is the first component to upgrade and the Service Manager R2 integration pack works with Service Manager SP1 too. The upgrade sequencing for System Center 2012 R2

  1. Service Management Automation
  2. Orchestrator
  3. Service Manager
  4. Data Protection Manager (DPM)
  5. Operations Manager
  6. Configuration Manager
  7. Virtual Machine Manager (VMM)
  8. App Controller
  9. Service Provider Foundation
  10. Windows Azure Pack for Windows Server
  11. Service Bus Clouds
  12. Windows Azure Pack
  13. Service Reporting

More info about the upgrade here.

 

Note that this is provided “AS-IS” with no warranties at all. This is not a official support statement.

Compare Software on Computers

From time to time you need to compare what software are installed on two computers. For example when using a test environment to test a new release of a software you want to make sure the test environment has same version installed as the production environment. Service Manager includes all this information as it synchronize it from Configuration Manager and Operations Manager. Unfortunately there is no out of the box report or feature to do this. I guess we could do it with Service Manager Powershell but for an Orchestrator geek like me it is much fast done with a runbook J

Figure 1 shows the master runbook. It invokes three other runbooks, one to gather all installed software, one to compare installed software and one to find all software installed on both computers. The solution use a temporary SQL table to store data. The result, a report, is sent to the Service Request “requester” by e-mail. In the start of the master runbook there are three Service Manager Activities, these are used to get the “requester”. The Send Email activity in the end use this information to send the report file.

Figure 2 shows runbook 8.3.2. This runbook first drops the temporary table if exists. The temporary table is named based on the computer name, if we have ran a comparison before, we delete the temp table. The master runbook will also delete temporary table, but if there is a “left over” from another job it is deleted in the 8.3.2 runbook and also we want to make sure no old data is affecting our new report. The 8.3.2 runbook gets all the installed software on one computer and insert it into the temporary SQL table. The runbook insert both software display name and GUID. Two software could be named the same, so then we also compare GUID of the CI. If we were interesting in installed patches we could use Software Updates class instead of the Software Items class. We run runbook 8.3.2 once of each computer.

Figure 3 shows runbook 8.3.2. This runbook use a SQL query to find software installed on one computer but not the other one. We run runbook 8.3.2 once for each computer.

Figure 4 shows runbook 8.3.4. This runbook use a SQL query to find software installed on both computers.

 

Figure 1. Master runbook

 

Figure 2. Runbook 8.3.2

Figure 3Runbook 8.3.3

Figure 4 Runbook 8.3.4

 

I use Service Manager Portal to submit a service request which will start a runbook activity, runbook 8.3.1. Figure 6 shows an example report, delivered by e-mail. The report is very basic, built in HTML. You could re-design it easy by changing HTML code in the runbook.

Figure 5

Figure 6 Report

 

You can download the runbooks, CompareSoftware. Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.

Build your own write to log activity

In a number of blog posts I have been writing about how to create a log for your runbooks. The solution I like the most is to write to a SQL database. Even if you do that there is always a small risk that some colleagues don’t write the log in the same way. For example someone use 1-2-3 instead of low-medium-high when writing severity. One way to handle that is data validation in your log runbook. Another way is to create a new activity where colleagues can pick from menus.

First thing you need to build a “write to log” activity is a Powershell script that do the “write to database” part.


param($Argument1,$Argument2,$Argument3)
$DESCRIPTION = $Argument3
$DESCRIPTION = $description.replace("'","''")
$DATE = get-date -format s
$RUNBOOK = $Argument1
$SEVERITY = $Argument2
$conn = New-Object System.Data.SqlClient.SqlConnection
$conn.ConnectionString = "Data Source=SCO12SP1-01;Initial Catalog=OrchestratorTool;Integrated Security=SSPI;"
$conn.open()
$cmd = New-Object System.Data.SqlClient.SqlCommand
$cmd.connection = $conn
$cmd.CommandText = "INSERT INTO Log (DATE,RUNBOOK,SEVERITY,DESCRIPTION)
VALUES ('$DATE','$RUNBOOK','$SEVERITY','$DESCRIPTION')"
$cmd.executenonquery()
$conn.close()

As you can see the SQL server name (SCO12SP1-01) and the SQL database name (OrchestratorTool) is hardcoded into the script. This is because that will not change, when using this activity we will always write to the same log database. Once the script is ready we can use the Orchestrator Integration Toolkit to build an activity and an integration pack. When configure the activity, configure it to run the Powershell script and pass the arguments.

20131009_WriteLogAct01When building the integration pack make sure to attach the PowerShell script. The Powershell script will be deployed together with the integration pack to all your runbook servers. Note that the Severity parameter is configured as Selection. It is configured as a drop down menu where users can pick severity from a list.

20131009_WriteLogAct02When using the activity you can pick severity from a menu. Runbook Name is published by all activities by default.

20131009_WriteLogAct03

You can download the integration pack and example files here, NOT SUPPORTED. Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.

The System Center 2012 Orchestrator book has arrived

The book has finally arrived in printed format. I hope it will provide you with good information and ideas around Orchestrator. Thanks again to the team behind the book, has been great working with you. You can order the book at for example Amazon, or if you are in Sweden maybe Adlibris is easier. Happy reading!

WP_20131003_025

Create Event Integration Pack

Some weeks ago someone asked me how to specify event ID, source and log name with the standard Send Event Log Message activity. Unfortunately the standard Send Event Log Message don’t provide a way to specific more than computer, message and severity. The event will always be generated with event ID 1 and source equals Orchestrator Runbook. The event is always written in the Application log.

To provide a way to specific more settings I created a new Create Event activity today. The activity is built around the EventCreate command. EventCreate enables an administrator to create a custom event in a specified event log, more info at Technet.

You can download the integration pack here. Note that this is provided “AS-IS” with no warranties at all. This is not a production ready management pack or solution, just an idea and an example.

Dashboard in SharePoint without any System Center database schema knowledge

 

Some time ago I was working in a project were we needed a dashboard to display data from multiple System Center components. You can often find a dashboard to download with pre-populated SQL queries, but it works for one component at a time. We needed one dashboard to work with multiple components and different databases, and we didn’t wanted to write any SQL queries against the System Center databases. Writing SQL queries to get the right data from the System Center databases can be a very time consuming work. The solution was to use the “old classic” Configuration Manager Dashboard that I have blogged about before, here and here, together with Orchestrator as a connector. We also created a small database in the middle. The dashboard database.

 

 

A number of runbooks in Orchestrator gets data from System Center components and from a custom database. The result is written to the dashboard database. The SharePoint dashboard queries the dashboard database. We created a set of tables in the dashboard database, for example one table for incidents, one for service requests, one for VM and one for Operations Manager alerts. The table for Service Manager had cells to store information like number of total number of open incidents, closed incidents today, new incidents for today and number of incidents with top severity and priority. Each cell in the table was updated by separate runbooks in Orchestrator, as not all data needed to be updated on the same frequency.

The runbook in figure 1 show a runbook that gets number of open alerts in Operations Manager and writes that to the dashboard database. As you can see we use a counter to count number of alerts. The Get Alert activity don’t publish a “number of object” value, so instead we have to do the counting in the runbook.

Figure 1

Figure 2 show a runbook that gets number of incidents created today in Service Manager and writes that to the dashboard database. The Get Object activity publish a Number of Objects value that we write to the database, then we don’t need to use counters as in Figure 1. To avoid running the Write To Database activity multiple times the Get Open Incidents Created Today activity is configured to flatten the result.

A nice benefit with using runbooks to get the data from the source system is that we don’t need to know anything about the source system database structure. We use only default activities from different System Center integration packs and Orchestrator standard activities.

Figure 2

Figure 3 show an example of the dashboard. There is a number of charts, gauges and tables you can use to present the data. Figure 4 show the configuration for one of the gauges. As we use a custom dashboard database with a very simple structure each SQL query is short and simple.

Figure 3

Figure 4

Note that this is provided “AS-IS” with no warranties at all. This is not a production ready management pack or solution, just an idea and an example.

State tracking and logging in Orchestrator

If you have an Orchestrator environment with multiple runbook servers might have notice that when a runbook failover to secondary runbook server the runbook restarts at the first activity. Let’s say you have a runbook that creates a new user account, a user folder and deploy a computer to a client. If that runbook is interrupted between creating the user folder and deploying the computer it will failover to secondary runbook server and restart at creating the user account. The issues with this is this, in this example, is that you will end up with multiple user accounts for the same user, or the runbook will crash as it tries to create the same user account again. In other scenarios it could cause multiple installation of patches, multiple restarts of a server or multiple deployed of the same VM.

In this blog post I will show a couple of alternatives to handle state tracking. I will also include an example on logging for your runbooks. In my examples I use a SQL table to store logs from all runbooks.

SQL
For all state tracking you need to have a unique ID of the runbook job. If the runbook only runs in one instance you could use the runbook name or activity ID, but often the same runbook can run in multiple instances at the same time.

First alternative is to use a SQL table. I often create a database called OrchestratorTool or something similar to store logs, temp data and so on, in this database I create tables for different runbooks. It is often complicated to use the same table for state tracking from multiple runbooks.

Figure 1 show a “master” or “parent” runbook that invokes three other runbooks. This runbook is used from Service Manager when order a new user account, it creates an account, a user folder and a workstation. The “master/parent” runbook is the runbook that we use as a runbook activity in Service Manager. The Service Request ID (SR) from Service Manager is the unquiet ID that these runbooks use to do state tracking. After each Invoke Runbook there is a check to make sure the runbook returns “something” and not NULL. In the end of the runbook it updates the service request in Service Manager with the result, for example account name. If any of the three Invoke Runbook activities returns NULL I want the runbook to stop, I use the Fail Runbook activity to do this. The Fail Activity runbook is a Query Database activity with incorrect configuration. This will make sure the runbook stops. All the “Write to log…” Invoke Runbook activities invoke a runbook that will write data to a log table in a SQL database. Figure 2 show configuration example of a “Write to log” Invoke Runbook activity. Figure 3 show the runbook that writes to the log table.

Figure 1

Figure 2

Figure 3

Figure 4, 5 and 6 shows the different runbooks that the “master/parent” runbook invokes. Each of these runbooks starts by checks if for example a folder has been created for the service request. If the folder is not created it will create a folder and update the tracking database table with info. If there is already a folder in the tracking database table for this service request it will publish that information back to the “master/parent” runbook. Each runbook also includes a couple of “Write to log” invoke runbook activities to write data to the log table in SQL. If any of these runbooks fails the error will be written to the log table and the “master/parent” runbook will fail, with the “Fail runbook” activity. As you can see in figure 2 the severity is CRITICAL, when a runbook sends CRITICAL severity to the log runbook (figure 3) it will generate an alert in Operations Manager with the info and write it to the log table. This means that each runbook don’t have to include “Create Alert” activities to generate an alert in Operations Manager, they just need to write a CRITICAL event to the log table.

Figure 4

Figure 5

Figure 6

With the SQL alternative the “master/parent” runbook can be restarted multiple times and each “child” runbook will check if has already ran for the service request. We use a table in a SQL database to store the state of each service request. We need another runbook to clean up the state table in SQL, or do it as a last step in the “master/parent” runbook.

Service Manager

The second alternative is to use Service Manager to store the state of the runbook. In this alternative each runbook is independent of each other, we don’t have a “master/parent” runbook, and instead each “child” runbook is its own runbook activity in Service Manager. Servicedesk operators can easy re-start or skip an activity if needed and customer can track each step. If you have only one huge runbook activity, invoke a “master/parent” runbook that then invokes 50 other runbooks, it is very difficult to troubleshoot or restart.

When we build runbooks that handle success, warning and failure from all activities the runbook always ends with a success. Figure 7 show an updated version of the runbook shown in figure 5, to create the account I Active Directory. If for example the Create User activity fails the runbook will go to the Write to log – Can’t Create Account activity. That activity will invoke a runbook and write information to the log. That invoke runbook activity will most likely work, so the runbook will end as a success even if it was not a success, we couldn’t create the account in Active Directory. That is the reason for the “Fail Runbook” activity, which will make sure the runbook crash. The Fail Activity runbook is a Query Database activity with incorrect configuration. This will make sure the runbook stops.

In the end of the runbooks shown in figure 7, 8 and 9 you can see an “Update RB Activity” activity. This activity will write the result of the runbook, for example an account, to TEXT1 of the runbook activity. Each instance of a runbook activity has a number of text fields that you can use to store information. In figure 8 you can see that we use “Get Related RB activities” based on the Service Request to get the runbook activity named something with 9.2.1. 9.2.1 Is the runbook that create the account, shown in figure 7. The runbook in figure 8 then readers TEXT1 from that runbook activity and use it as input when create the folder.

Figure 7

Figure 8

Figure 9

The log table I use for these examples looks like the table in figure 10. The SQL state tracking table I use for the first example looks like the table in figure 11.

Figure 10

 

Figure 11

Download my example runbooks, StateTracking.
Note that this is provided “AS-IS” with no warranties at all. This is not a production ready management pack or solution, just an idea and an example