Home » System Center Operations Manager 2007 (Page 2)

Category Archives: System Center Operations Manager 2007

Contoso.se

Welcome to contoso.se! My name is Anders Bengtsson and this is my blog about Azure infrastructure and system management. I am a senior engineer in the FastTrack for Azure team, part of Azure Engineering, at Microsoft.  Contoso.se has two main purposes, first as a platform to share information with the community and the second as a notebook for myself.

Everything you read here is my own personal opinion and any code is provided "AS-IS" with no warranties.

Anders Bengtsson

MVP
MVP awarded 2007,2008,2009,2010

My Books
Service Manager Unleashed
Service Manager Unleashed
Orchestrator Unleashed
Orchestrator 2012 Unleashed
OMS
Inside the Microsoft Operations Management Suite

Exchange mail flow check with Opalis and Operations Manager

The Exchange management pack includes a mail flow check, both between internal mail server and cross organization. With the management pack we get good monitoring of all the Exchange components and also that mail servers can communicate with each other. The challenge I see sometimes is that you want to verify e-mail from a client to a client, trough all mail server, spam filters, firewalls and Internet connections. Not only between mail servers, but between clients.

One way is with Opalis. Opalis can integrate with Operations Manager 2007 R2 and there is a community integration pack for Exchange. There are some legacy exchange objects in Opalis too, but you don’t want to use legacy objects. They are there only for backwards compatibility with old Opalis versions. This legacy category is hidden by default in the console, this is all so that new customers would refrain from trying those activities in fresh work flows. More info about legacy objects here.

The Exchange Opalis Extension can be downloaded here. This integration pack is built on Exchange Web Services and is compatible with Exchange 2007 SP1, Exchange 2010 and Exchange 2010 SP1. Please note that this integration pack is not from Microsoft and is not support and tested in the same way as integration packs included in the product.

In this example the policy sends a e-mail every 15 minutes. Then it waits 5 minutes and checks if it has received a answer. The recipient of the e-mail should be a echo mailbox that simple sends a reply pack. If it has received a reply it will empty the inbox and report OK. If not it will generate an alert in Operations Manager.

  1. Every 15 minutes. Trigger the policy to run every 15 minutes
  2. Send Exchange mail. This sends a e-mail. The recipient needs to be a mailbox that simple sends a reply direct.
  3. Wait 5 minutes. There is a trigger delay on this link that waits five minutes, this is the time we give the mail environment to recive the reply
  4. Find mail. This object checks in the inbox for a new e-mail from the echo mailbox. There is always a risk some other mail has been received, so the object makes sure it has the correct subject and from address
  5. Compare Values. This object verifies that the “Find mail” object did find a e-mail. If not it will generate an alert in Operations Manager as there is something not working in the mail flow. If there is a e-mail it will return true and the work flow will move on
  6. Empty Inbox Folder. We don’t want to fill up this test mailbox, therefor we clear the inbox
  7. Write OK to eventlog. We write a simple OK to the event log. Always good with a note that we ran the policy and it was working

This policy will send a e-mail as a client would have, to a e-mail address somewhere on the Internet. The recipient will reply instant and the policy will verify there is a reply received within five minutes. The really nice thing with a mail flow check like this is that you verify everything between the recipient mailbox and your “Opalis client” mailbox. When this policy returns “OK” we know the external mail flow is really working.

Override RunAs Logon Check Failed

The other day I needed to configure a web application check against a web service in a DMZ. This DMZ had its own Active Directory domain with no trust or connection to the internal domain where Operations Manager was installed. One of the criteria of the web application monitoring was to verify that we could log on to the web service. We ran into a challenge when configure the run as account for the web service. Operations Manager is trying to verify the account, as we didn’t have a trust or any connection to the DMZ domain that was not possible. We tried to override only for this account on the watch node, but unfortunately it was all account checks or none.

After some investigation we found that the event generated on the agent (the watch node in this case) included the account that did pass the check.

The solution to this was to disable the monitor for the watch node, then create a new monitor only for the watch node with the same criteria, but with one extra criteria, not to include events where event description includes LITWARE\Administrator. LITWARE\Administrator was the account that we didn’t want an alert on.

State based on logon status

I read a question this week about presenting a state based on if a service account was logged on to a machine or not. There are a couple of different ways to do that. We will need a monitor in both of them, as we want to present a state. You could solve it with a monitor running a script, checking current logged on users. You could also solve it with a monitor checking log-on and log-out events.

If you want to do it with two events and running server 2008 you could look at even 4624 (an account was successfully logged on) and 4647 (User initiated log-off). To configure a monitor like that you follow the steps below. This example will configure a monitor to generate an alert if not the CONTOSO\svc-contoso-service account is logged on.

  1. In the console, navigate to Authoring/Management Pack Objects
  2. Right-click monitors and create a new Simple Event Detection/Windows Event Reset monitor
  3. Choose a suitable management pack, not the default management pack. Click Next
  4. Input a name, a target and choose a parent monitor. Click Next
  5. Event Log (Unhealthy Event). Select Security as event log name. Click Next
  6. Event Expression (Unhealthy Event). Select Event ID equals 4647 and EventDescription Contains CONTOSO\svc-contoso-service. Click Next
  7. Event Log (Healthy Event). Select Security as event log name. Click Next
  8. Event Expression (Healthy Event). Select Event ID equals 4624 and EventDescription Contains CONTOSO\svc-contoso-service. Click Next
  9. Configure Health. Click Next
  10. Configure Alert. Check the “Generate alerts for this monitor” check box. Click Create
  11. By default this new monitor will be forced to healthy state first. This mean we will not get an alert until the account log off. In other words, if there is a machine where this account is not logged on right now, Operations Manager will not notice it. It is only looking for the log-in and log-off event. To generate an alert we first need to log-on with the account, in this example CONTOSO\svc-contoso-service, then log-out again. The monitor will then go into unhealthy state.

    The second way to solve this is with a script. To configure a monitor like that you follow the steps below

  1. In the console, navigate to Authoring/Management Pack Objects
  2. Right-click monitors and create a new Scripting/Generic/Timed Script Two State Monitor
  3. Choose a suitable management pack, not the default management pack. Click Next
  4. Input a name, a target and choose a parent monitor. Click Next
  5. Schedule. Configure how often you want the script to run, to see if the account is logged on. Dont configure it to run to often, check your organization requirements. Click Next
  6. Script. Input a script name, for example accountcheck.vbs. Paste the script into the script text field. You can download the script here. One of the first lines in the script configure which account to check. Don’t forget to update it. Click Next
  7. Unhealthy Expression. Input
    Property[@Name=’Status’] Does not equal OK
    Click Next
  8. Healthy Expression. Input
    Property[@Name=’Status’] Equals OK
    Click Next
  9. Health. Click Next
  10. Configure Alert. Check the “Generate alerts for this monitor” check-box and input a alert description. Click Create

If you need to display the account status in a distributed application it is easiest if you can target the monitor to a new customer class. The health model of the class will then be only this monitor, in other words the class instance health it will only show the status of the account.  

Thanks to Guy Thomas for his script that I adapted.

Send e-mail to affected user

In this post I will show you how to build a simple task to send a e-mail to the affected user of a incident. The e-mail will be logged in the action log of the incident.Please note that this will not affect how you handle incoming e-mails, it will only give you a fast way to send a e-mail to the affected user.

  1. In the SCSM Console, go to Library, check Tasks
  2. In the Tasks pane click Create Task
  3. Input the following settings in the “Create a task” wizard
  4. General
    • Task name: Contoso – Send e-mail to affected user
    • Target class: Incident
  5. Categories: Incident Support Group Folder Tasks
  6. Command Line
    • Full path to command: %SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe
    • Parameters: .\emailenduser.ps1 Property/WorkItem/ID Property/AffectedUser/Username Property/WorkItem/Title Property/AffectedUser/Firstname
    • Please note that each property is inserted with the “Insert Property” button
    • Check both “Log in action log when this task is run” and “Show output when this task is run”
    • Working Directory: C:\Scripts\
  7. Save your new task

As you can see in the task we are using a script here, C:\Scripts\emailenduser.ps1. It is a powershell script that you can download here, emailenduser. Please note that you will need to modify the script to work in your environment, there are some notes about that in the script.

Now, if you select a incident in a incident view you will have a new task in the tasks pane, “Contoso – Send e-mail to affected user”. If you click it you can input a text. This text will be in the e-mail. The e-mail will also add a “Hi affect user name” and “thanks, br service desk…”. Don’t forget to modify that part to in the script.

The operator will see the e-mail and it will also be inserted into the action log of the incident.

Show all agents without ACS enable

In Operations Manager 2007, Audit Collection Services (ACS) provides a means to collect records generated by an audit policy and store them in a centralized database. More info about ACS here.

If you enable ACS gradually or as you install agents it is easy to forget a agent or two. Then you need a view that shows you all agents where you have not enable ACS. I was looking for that kind of view a couple of days ago. Unfortunaly I could not find it so I created it myself instead. My idea was to show all machines where the AdtAgent service, the ACS agent service, is disable. To get this to work we need to do a discovery of a registry value on all machines with a health service. The discovery will tell us the start-up configuration of the AdtAgent service (ACS agent). The result of the discovery will shown as a extra property of the health service.

  1. In the Operations Manager console navigate to the Authoring workspace.
  2. Expand management pack objects and right-click attributes, select Create a New Attribute
  3. In the Create Attribute Wizard, General Properties page, input a name for example Contoso – noACSAgent
  4. In the Create Attribute Wizard, Discovery Method page, input
    • Discovery Type: Registry
    • Target: Health Service
    • Management Pack: Select one of your unsealed MPs or create a new MP
  5. In the Create Attribute Wizard, input
    • Key or Value Type: Value
    • Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\AdtAgent\Start
    • Attribute Type: Int
  6. Click Finish to save your new attribute
  7. Navigate to the Monitoring workspace, right-click the management pack where you stored the attribute, select create new state view
  8. Name the view “Contoso – Agents without ACS” and configure
    • Show data related to: Health Service_Extended
    • with specific Contoso – noACSAgent , configure it with 4 (Show all state with 4 Contoso – noACSAgent) A service configured with startup to 4 means that the service is disable.
  9. Click OK to save your new view

This view will now show you all agents with a disabled ACS (AdtAgent) service.

Monitor the internet connection as a client

“Hello, is it IT support? My Internet is slow”

Have you heard that before? Last week a customer asked me if we could monitor how long it takes to download a file from a client workstation, and present it in a nice performance view in the Operations Manager console. Of course we can, with a script in a rule. The script downloads a file from Internet, measure the time it takes and reports it back to Operations Manager as performance data. You can control which machines that runs the script, so you cant select some workstations or machines in remote offices. If it takes more then 3 seconds it will generate a local event. If there is a problem downloading the file it will also generate a local event. You can configure Operations Manager to pickup this event and generate an alert. Thanks to Patrik the script can detect if the file we are trying to download is missing or if there are any other download related problems. I have attached the script I used, and below is the steps that you need to take to implement it.

To use this script, follow these steps

  1. In the Operations Manager console, navigate to the Authoring workspace, right-click Rules and select Create New Rule
  2. In the Create Rule Wizard, select to create a Probe Based/Script (Performance) rule, select a suitable management pack for example Contoso – Internet Download
  3. On the Rule Name and Description page, input a rule name for example “Contoso – Internet Download – Download 20 Mb file”. Select a suitable target for example Windows Server. Uncheck the “Rule is enabled” box.
  4. On the Schedule page, input how often you want to script to run. For example every 10 minutes
  5. On the Script page, clear the script text box and paste the script from the attached file. Configure the timeout to 5 minutes. Change the script name to ContosoDownloadscript.vbs
  6. On the Performance Mapper page, input
    • Object: 20MB.zip
    • Counter: FileDownload_Time
    • Instance: select Netbios computer name from the menu
    • Value: $Data/Property[@Name=’Contoso_PerfValue’]$
  7. Create the rule

Now you have a rule target to all Windows Servers. But it is disabled. To control which machine that is running this, create a new group, save it in the same management pack (important!) and add some windows servers as explicit members. These machines will run the script and download the file.

  1. In the Operations Manager console, navigate to the Authoring workspace, right-click Groups and select Create a new Group
  2. In the Create Group Wizard, input a group name, for example “Contoso – Internet Download nodes”. Select the same management pack as you used before, in my case Contoso – Internet Download
  3. On the Explicit Member page, click Add/Remove Objects
  4. In the search for drop down menu, select Windows Server. Then in the name field input the machine or machines that you want to use as watcher node and add then under selected objects
  5. Click next a couple of times in the wizard and then save and close it

To enable the rule against the machines in this group,

  1. Find the rule again under Authoring and rules
  2. Right-click it and select Overrides, Override the Rule, For a group, select your group
  3. In the Override properties window, select enable and configure the override value to TRUE. Click OK to save.

The next step will be to configure a rule to pickup the event. These are the two events that you want to generate alert on

  1. Navigate to Authoring workspace, right-click Rules, select Create a new rule…
  2. In the Create Rule Wizard, select to create a Alert Generating Rule, Event Based, NT Event Log (Alert). Select the same MP again, in my case Contoso – Internet Download
  3. On the General page, input a name, for example Contoso – Internet Download – Event rule. Select “Windows Server” as rule target. Uncheck the “Rule is enabled” check box
  4. On the Event Log Type page, select Application
  5. On the Build Event Expressen, input
    • Event ID Equals 2
    • Event Source Equals WSH
  6. On the Configure Alerts page, click Alert suppression…
  7. In the Alert Suppression window, select Event ID, Logging Computer, Event Source, click OK
  8. On the Configure Alerts page, click Create

Now we need to enable this for the machines that we use as watcher nodes. As we created this rule with “Rule is enabled” unchecked, no machines is running it right now.

  1. Find the rule again under Authoring and rules
  2. Right-click it and select Overrides, Override the Rule, For a group, select your group
  3. In the Override properties window, select enable and configure the override value to TRUE. Click OK to save.

The last step could be to create the performance view

  1. In the console, navigate to the Monitoring workspace
  2. Select the folder for your MP, in my case “Contoso – Internet download”. Right-click it and select New > Performance View
  3. In the Properties window input
    • Name: Contoso – Download speed
    • Show data contained in a specific group: Select your group, in my case “Contoso – Internet download nodes”
    • Object: 20MB.zip
    • Counter: FileDownload_Time
  4. Click Ok to save you diagram view

If you want to change the file that we download, or the destination folder, you need to modify the script and the lines highlighted below

If you want to change the default 5 seconds threshold on the download, you need to modify the following line

Download the script here, Internetdownload

Notification popup with Opalis

Today I needed to build some kind of extra notification when a special alert was generated in Operations Manager. As we were already working with Opalis, why not continue with Opalis? The result was a simple policy checking for a special alert (service stopped), if it is generated, it sends a popup message with msg.exe to a specific machine. Pretty simple, but another example how to solve things with Opalis.

There was another policy checking the service and restared it if needed, of course we could merge them into one policy. The “Notify by popup” is a “Run program” object with the following settings

  • Program execution
  • Computer: IP of target machine where to run the program
  • Program path: C:\WINDOWS\system32\msg.exe
  • Parameters: * /SERVER:NYSRV10 /TIME:120 {Name from “Get All New Alerts”}
  • Working folder: C:\

The message on the target machine

…and the updated alert in Operations Manager, by the second policy.

Export agents to Excel

A common questions in the forums is “how to export a list of all agents to Excel?” You can do that with Operations Manager shell. The Operations Manager command shell is built on Windows PowerShell. The Operations Manager command shell extends the Windows command shell with over 80 additional small utility programs, called cmdlets, which can either be run directly from the command shell prompt or called from within a batch file or script. To export a list of all agents and some info about them run the following command

get-agent | select Displayname,InstallTime,InstalledBy,Version | export-csv -notype c:\agents.csv

That will give you a CSV file, C:\agents.csv. Open the file with Excel, select all the data, click the “text to columns” button and choose comma delimeter. That will convert your data into nice columns.

…and there you have your list in Excel with nice columns

Check Last Line Only

I wrote a script to check only the last line of a file. The scripts checks the last line every time it run. If you search my blog you will find a number of script to read logfiles. Create a new two state script monitor where you include the script below.  In my example script I looks in the C:\temp\myfile.txt file for the word “Warning” ( varWarPos = Instr(strLine, “Warning”) )

  • Unhealthy Expression
    • Property[@Name=’Status’] Contains warning
  • Healthy Expression
    • Property[@Name=’Status’] Contains ok
  • Alert description
    • You could write any alert description here, but if you include the following parameters you will see the whole line and the status in the alert description.
    • State $Data/Context/Property[@Name=’Status’]
    • Line $Data/Context/Property[@Name=’Line’]$
Set oAPI = CreateObject(“MOM.ScriptAPI”)
Set oBag = oAPI.CreatePropertyBag()
Const ForReading = 1
strFile = “C:\TEMP\myfile.txt”
Set FSO = CreateObject(“Scripting.FileSystemObject”)
Set inFile = FSO.OpenTextFile(strFile)
lines = Split( inFile.ReadAll, vbLF )
lineCount = UBound(lines)
Set FSO = CreateObject(“Scripting.FileSystemObject”)
Set TextFile = FSO.OpenTextFile(strFile, ForReading)
For i = 1 to lineCount
TextFile.ReadLine
Next
strLine = TextFile.ReadLine
varWarPos = Instr(strLine, “Warning”)
If varWarPos > 0 Then
varStatus = “Warning”
varLine = strLine
End If
TextFile.Close
If varStatus = “Warning” Then
Call oBag.AddValue(“Line”, varLine)
Call oBag.AddValue(“Status”,”warning”)
Call oAPI.Return(oBag)
Else
Call oBag.AddValue(“Status”,”ok”)
Call oAPI.Return(oBag)
End If

Cross-platform failover

With default settings cross-platform agents in Operations Manager 2007 R2 don’t fail-over to another management server if the primary management server goes offline. If the primary management server goes offline you will loose monitoring of all cross-platform machines communicating through that management server. The System Center Cross platform and Interop team blog has a couple of scripts that can help you manage cross-platform machines. One of these scripts is used to to change the current management server monitoring a UNIX/Linux server (or group of UNIX/Linux servers) to a new management server.

But if you move a cross-platform over to another management server you will get an alert like this

A simple explanation to this is that the new management server don’t trust the CA which have issue the certificate that the cross-platform machine is communicating with. You can solve this by copy the certificate from the management server that first discover the cross-platform machine over to the second management server. Store the certificate under Local Computer/Trusted Root Certification Authorities (in the certificates MMC). The certificate is named SCX-Certificate, issued by the primary management server.

 

When the new management server has the certificate installed it will start communicate with the cross platform machine. You could build a timed script two state monitor to check if the first management server is working, else run the script to move all cross-platform agents.