Add support for MSOL Proxy, custom Rules, and $OCCAM environment vari…

…ables (#1)

* initial work proxying MSOnline cmdlets

Adds functionality that dyanmically builds an MSOnline proxy module that wraps each MSOnline function, strips the `-TenantId` param, and auto-fills it so that rule evaluation can just use the `Get-MsolUser` or similar cmdlets and have it automatically scoped to the tenant ID

* Add support for MSOnline proxying and $OCCAM environment variables

* Refactor MSOL Proxy as a dynamic in-memory module

The MSOL proxy module previously worked by saving all portions of the module to disk and then importing it. This required clean up after the fact and was unneccessary. Now, it uses PowerShell's Dynamic Module feature to store everything in memory.

* Add rule for finding users with non-default authentication policy

also remove old reference to cleaning up tmp directory

* Add support for custom rule files

OCCAM now parses any files ending in `.Rule.ps1` while building the ruleset

* Update auth policy users rule

* Update Find-ExplicitAuthPolicyUsers.Rule.ps1

Made more robust

.gitignore

@@ -0,0 +1,4 @@
+.DS_Store
+Thumbs.db
+
+Office365**/

README.md

@@ -37,22 +37,52 @@ The output will also be saved to a CSV with an execution timestamp.
 
 A set of pre-made best practices have been bundled with this module. They include:
 
-1. `Test-BasicAuthPolicies` - Checks to ensure that authentication policies block basic authentication mechanisms
-2. `Test-PopImap` - Checks for any users that have POP or IMAP enabled and exports a CSV list of them
-3. `Test-UnifiedAuditLogging` - Checks that Unified Audit Logging is enabled on the tenant
+1. `Find-ExplicitAuthPolicyUsers` - Finds any users not using the organization's implicit authentication policy and (if any) exports the list as CSV
+2. `Test-BasicAuthPolicies` - Checks to ensure that authentication policies block basic authentication mechanisms
+3. `Test-PopImap` - Checks for any users that have POP or IMAP enabled and exports a CSV list of them
+4. `Test-UnifiedAuditLogging` - Checks that Unified Audit Logging is enabled on the tenant
 
-Additional rules can be added by inserting a compliant rule file into the `Rules` directory in the module. Rulesets are dynamically evaluated at run-time.
+Additional rules can be added by creating a file ending in `.Rule.ps1` in the directory from which you invoke OCCAM. Rulesets are dynamically evaluated at run-time.
+
+If you wish to ignore all default rules entirely, you can use the `-NoDefaultRules` switch:
+
+```ps1
+Invoke-Occam -NoDefaultRules
+```
 
 ## Writing Custom Rules
 
 A Rule is an arbitrary PowerShell script enriched with metadata that returns a hashtable of boolean pass/fail values. Albeit simple, Rules are flexible and powerful - anything you can write in PowerShell can be packaged as a Rule and evaluated against every Office365 tenant you manage.
 
-Rules are `.ps1` files expected to have the same name as the function contained within them. Any `.ps1` files in the `Rules` module directory are dynamically built into a RuleSet on runtime and evaluated.
+Any files ending in `.Rule.ps1` in the working directory are discovered and parsed automatically. The name of a `.Rule.ps1` file is expected to have the same name as the function contained within them (e.g., `Test-Something.Rule.ps1` is espected to have a function named `Test-Something` inside).
+
+If a name conflict is found between a custom Rule and a default Rule, the custom Rule takes precedence.
 
 ### Rule Execution Environment
 
 Rules are ran in an environment that has the [MSOnline](https://docs.microsoft.com/en-us/powershell/module/msonline/) and [ExchangeOnlineManagement](https://docs.microsoft.com/en-us/powershell/exchange/exchange-online-powershell-v2) modules pre-loaded and authenticated to the given tenant the Rule is being evaluated against. All cmdlets and functions in those modules are available for immediate use.
 
+There is no need for MSOnline commands to use the `-TenantId` parameter, as this value is dynamically injected with the ID of the tenant being audited. This means that you can call `Get-MsolUsers` or related functions and it will automatically return a collection scoped to the desired tenant!
+
+### Environment Variables
+
+OCCAM exposes custom environment variables that are available for use in your custom Rules. They are in the same form as the built-in PowerShell `env:` drive, and can be accessed accordingly:
+
+```ps1
+Write-Host $OCCAM:TenantName
+```
+
+The following OCCAM environment variables are avalable for use:
+
+| Variable                 | Description                                                 | Example                                                                                      |
+|--------------------------|-------------------------------------------------------------|----------------------------------------------------------------------------------------------|
+| $OCCAM:TenantName        | Office 365 Tenant Name                                      | Contoso Corp                                                                                 |
+| $OCCAM:TenantId          | Tenant ID (GUID format)                                     | `b3d628ab-3271-4cc5-bd84-ce69d0946ec6`                                                         |
+| $OCCAM:TenantDomain      | Tenant's Primary Domain                                     | contoso.onmicrosoft.com                                                                      |
+| $OCCAM:RuleName          | Name of the rule currently being evaluated                  | Test-UnifiedAuditLogging                                                                     |
+| $OCCAM:OutputDir         | Output directory scoped to current tenant and rule          | `Office365 Security Audit - 2021-01-13_15_18_06/Contoso Corp/Test-UnifiedAuditLogging` |
+| $OCCAM:AuthenticatedUser | User Principal Name of the account used for Exchange Online | steve@example.com                                                                            |
+
 ### Rule Output
 
 Rules are expected to return a hashtable of key/value pairs corresponding to the test case(s) the Rule evaluates. Each value is expected to be a boolean, as Rules are meant to evaluate to a simple Pass/Fail criteria.
@@ -64,7 +94,21 @@ Rules are expected to return a hashtable of key/value pairs corresponding to the
 }
 ```
 
- If more robust information is needed (e.g., a list of authentication policies with Basic Auth enabled), it is suggested to export that information as a CSV.
+### Exporting as CSV
+
+If more robust information is needed (e.g., a list of authentication policies with Basic Auth enabled), it is suggested to export that information as a CSV. Any files that a rule generates should be created by using the `$OCCAM:OutputDir` relative path that is provided. The `$OCCAM:OutputDir` path is unique for each invocation, tenant, _and_ rule, and it follows the following format:
+
+```txt
+<directory with invocation timestamp>\<Office 365 Tenant Name>\<Rule Name>
+```
+
+Your Rule is responsible for creating the directory path using `New-Item`. The following example snippet gets a list of users and exports it as CSV. Assuming a customer name of `Contoso Corp` and a rule name of `Test-MyCustomRule`, the following code would generate a CSV file at `./Office365 Security Audit - 2021-01-13_15_18_06/Contoso Corp/Test-MyCustomRule/users.csv`.
+
+```ps1
+$Users = Get-MsolUsers
+New-Item -ItemType Directory -Force -Path $OCCAM:OutputDir | Out-Null
+$Users | ConvertTo-Csv -NoTypeInformation | Out-File ('{0}/users.csv' -f $OCCAM:OutputDir) -Force
+```
 
 ### Rule Metadata
 

occam/Internal/Build-MsolProxy.ps1

@@ -0,0 +1,48 @@
+function Build-MsolProxy {
+  [CmdletBinding()]
+  param (
+      [Parameter(Mandatory = $true)]
+      [System.Guid]$TenantId
+  )
+
+  $commands = Get-Command -Module MSOnline
+
+  $functionsToExport = @()
+
+  foreach ($command in $commands) {
+    $metadata = New-Object System.Management.Automation.CommandMetaData $command
+
+    # remove TenantId from command metadata
+    $hasTenantIdParam = $metadata.Parameters.ContainsKey("TenantId")
+    if ($hasTenantIdParam) {
+      $metadata.Parameters.Remove("TenantId") | Out-Null
+    }
+
+    # create a proxy function that wraps the initial MSOnline cmdlet
+    $proxy = [System.Management.Automation.ProxyCommand]::Create($metadata)
+
+    # string-replace the PSBoundParameters splat operation to insert the -TenantId parameter
+    # into the underlying command being called/wrapped
+    if ($hasTenantIdParam) {
+      $proxy = $proxy -replace '@PSBoundParameters', ('@PSBoundParameters -TenantId {0}' -f $TenantId)
+    }
+
+    # Pack the internals as a function
+    $proxyAsFunction = "function $($command.Name) { `n $proxy `n }"
+
+    # Append the full proxy function as a string onto an array
+    $functionsToExport += $proxyAsFunction
+  }
+
+  # Concatenate all functions into one large string with new lines separating each
+  $ScriptString = ($functionsToExport -join("`n"))
+
+  # Convert the string to a scriptblock
+  $ScriptBlock = [Scriptblock]::Create($ScriptString)
+
+  # Load the proxy functions as a dynamic module into memory, and pipe to
+  # the Import-Module command so we can clean it up with Remove-Module later
+  New-Module -Name "MSOL_$TenantId" -ScriptBlock $ScriptBlock | Import-Module
+
+  return "MSOL_$TenantId"
+}

occam/Internal/Build-RuleSet.ps1

@@ -1,24 +1,40 @@
 function Build-RuleSet {
+  [CmdletBinding()]
+  param (
+      [Parameter()]
+      [Switch]$NoDefaultRules = $false
+  )
   $ModuleBase = $MyInvocation.MyCommand.Module.ModuleBase
-  $RulePath = "$ModuleBase\Rules"
+  # $ModuleBase = ".\occam"
+  $DefaultRulePath = "$ModuleBase\Rules"
 
   # Populate rule name list by grabbing any files in the Rules directory
-  $Rules = Get-ChildItem -Path "$RulePath" -Filter *.ps1 -Recurse | ForEach-Object { $_.BaseName }
+  $Rules = @()
+
+  # Discover additional rules in execution path
+  $Rules += Get-ChildItem -Filter *.Rule.ps1 -Recurse
+
+  # Get default (prepackaged) rules
+  if (!$NoDefaultRules) {
+    $Rules += Get-ChildItem -Path "$DefaultRulePath" -Filter *.Rule.ps1
+  }
+
+  $Rules = $Rules | Sort-Object -Property Name -Unique | ForEach-Object {@{ Name = ($_.Name.split(".") | Select-Object -First 1); Path = $_.VersionInfo.FileName  }}
 
   $FormattedRuleSet = @()
 
   $i = 0
   foreach($Rule in $Rules) {
-    Write-Progress -Activity "Building Rule Set" -PercentComplete ($i / $Rules.count * 100) -CurrentOperation "Importing Rule $Rule"
+    Write-Progress -Activity "Building Rule Set" -PercentComplete ($i / $Rules.count * 100) -CurrentOperation "Importing Rule $($Rule.Name)"
     # Load the rule as a module
-    Import-Module "$RulePath\$Rule.ps1" -Force
+    Import-Module $Rule.Path -Force
 
-    $RuleHelp = Get-Help $Rule
+    $RuleHelp = Get-Help $Rule.Name
     $FormattedRuleSet += @{
-      Name = $Rule;
+      Name = $Rule.Name;
       OutputKeys = $RuleHelp.returnvalues.returnValue.type.name.Split([Environment]::NewLine);
       Synopsis = $RuleHelp.SYNOPSIS;
-      Path = "$RulePath\$Rule.ps1"
+      Path = $Rule.Path
     }
     $i++
   }

occam/Internal/Invoke-TenantAudit.ps1

@@ -2,17 +2,18 @@ function Invoke-TenantAudit {
     param (
         [Object]$tenant,
         [String]$UPN,
-        [Object]$RuleSet
+        [Object]$RuleSet,
+        [String]$ReportPath
     )
-    $totalSteps = $RuleSet.length + 1
+    $totalSteps = $RuleSet.length + 5
     $i = 0
 
     $FormattedTenant = @{
         Name = $tenant.Name
     }
 
     Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete ($i / $totalSteps * 100) -CurrentOperation "Connecting to Exchange Online"
-    # Connect to client's Exchange Online and Security & Compliance Center using EXO V2
+    # Connect to client's Exchange Online using EXO V2
     try {
         Connect-ExchangeOnline -UserPrincipalName $UPN -DelegatedOrganization $tenant.Domain -ShowBanner:$false -ShowProgress:$false
         # Connect-IPPSSession -UserPrincipalName $UPN -DelegatedOrganization $tenant.Domain -ShowBanner:$false -ShowProgress:$false
@@ -23,10 +24,23 @@ function Invoke-TenantAudit {
         return New-Object PSObject -Property $FormattedTenant
     }
 
-    $i = 0
+    # Clean up MSOnline proxy module
+    $i++; Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete ($i / $totalSteps * 100) -CurrentOperation "Dynamically Generating MSOnline Proxy Module"
+    $MsolProxyModule = Build-MsolProxy -TenantId $tenant.id
+
+    # Generate PS Drive
+    $i++; Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete ($i / $totalSteps * 100) -CurrentOperation "Creating Runtime Environment Variables"
+    New-PSDrive -Name "OCCAM" -PSProvider Environment -Root . | Out-Null
+    $OCCAM:TenantName = $tenant.Name
+    $OCCAM:TenantId = $tenant.id
+    $OCCAM:TenantDomain = $tenant.Domain
+    $OCCAM:AuthenticatedUser = $UPN
+
     foreach ($Rule in $RuleSet) {
-        $i++
-        Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete ($i / $totalSteps * 100) -CurrentOperation $Rule.Synopsis
+        # Set Rule-specific environment variables
+        $OCCAM:OutputDir = "$ReportPath/$($tenant.Name)/$($Rule.Name)"
+        $OCCAM:RuleName = $Rule.Name
+        $i++; Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete ($i / $totalSteps * 100) -CurrentOperation $Rule.Synopsis
         try {
             Import-Module $Rule.Path -Force
             $output = Invoke-Expression $Rule.Name
@@ -37,12 +51,18 @@ function Invoke-TenantAudit {
         }
     }
 
-    $i++
-    Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete ($i / $totalSteps * 100) -CurrentOperation "Disconnecting from Exchange Online"
+    # Clean up PS Drive
+    $i++; Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete ($i / $totalSteps * 100) -CurrentOperation "Cleaning up Runtime Environment Variables"
+    Remove-PSDrive -Name "OCCAM" -Force
 
+    # Clean up MSOnline proxy module
+    $i++; Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete ($i / $totalSteps * 100) -CurrentOperation "Removing MSOnline Proxy Module"
+    Remove-Module -Name $MsolProxyModule -Force
+
+    # Disconnect from Exchange Online
+    $i++; Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete ($i / $totalSteps * 100) -CurrentOperation "Disconnecting from Exchange Online"
     Disconnect-ExchangeOnline -Confirm:$false *> $null  
 
     Write-Progress -Activity ("Auditing {0}" -f $tenant.Name) -ParentId 1 -PercentComplete 100
-
     return New-Object PSObject -Property $FormattedTenant
 }

occam/Public/Invoke-Occam.ps1

@@ -1,11 +1,16 @@
 Function Invoke-Occam {
+    [CmdletBinding()]
+    param (
+        [Parameter()]
+        [Switch]$NoDefaultRules = $false
+    )
     Begin {
         # Load Rules
-        $RuleSet = Build-RuleSet
+        $RuleSet = Build-RuleSet -NoDefaultRules:$NoDefaultRules
 
         # Create a folder to hold all results in, and add a timestamp for uniqueness
-        $dirname = New-Item -Name ('Office365 Security Audit - {0}' -f (get-date -f yyyy-MM-dd_HH_mm_ss)) -ItemType "directory"
-        $dirname = $dirname.Name
+        $ReportPath = New-Item -Name ('Office365 Security Audit - {0}' -f (get-date -f yyyy-MM-dd_HH_mm_ss)) -ItemType "directory"
+        $ReportPath = $ReportPath.Name
     }
     Process {
         $UPN = Read-Host -Prompt "Please enter your CSP email"
@@ -39,16 +44,19 @@ Function Invoke-Occam {
         foreach ($selectedTenant in $selectedTenants) {
             $tenant = $tenants | Where-Object {$_.Name -eq $selectedTenant}
             Write-Progress -Activity "Auditing Tenants"  -Id 1 -PercentComplete ($i / $selectedTenants.count * 100)
-            $formattedTenants += Invoke-TenantAudit -tenant $tenant -UPN $UPN -RuleSet $RuleSet
+            $formattedTenants += Invoke-TenantAudit -tenant $tenant -UPN $UPN -RuleSet $RuleSet -ReportPath $ReportPath
             $i++
 
         }
         Write-Progress -Activity "Auditing Tenants" -Id 1 -PercentComplete 100
 
-        $Properties = (,"Name" + $RuleSet.OutputKeys)
+        # Strip empty values out of the array so that only valid keys are sent to the Select-Object cmdlet
+        $RuleOutputKeys = $RuleSet.OutputKeys | Where-Object { $_ }
+        # Prepend "Name" so that it shows up first
+        $Properties = (,"Name" + $RuleOutputKeys)
         $formattedTenants = $formattedTenants | Select-Object -Property $Properties
         $formattedTenants | Write-PSObject -MatchMethod Exact -Column *, * -Value $false, $true -ValueForeColor Red, Green
-        $formattedTenants | ConvertTo-Csv -NoTypeInformation | Out-File ('./{0}/results.csv' -f $dirname)
+        $formattedTenants | ConvertTo-Csv -NoTypeInformation | Out-File ('./{0}/results.csv' -f $ReportPath)
     }
     End {}
-}
+}

occam/Rules/Find-ExplicitAuthPolicyUsers.Rule.ps1

@@ -0,0 +1,43 @@
+ <#
+.SYNOPSIS
+Report users assigned an explicit authentication policy
+
+.OUTPUTS
+#>
+function Find-ExplicitAuthPolicyUsers {
+  param ()
+  Begin {
+    $Users = @(Get-User -ResultSize Unlimited)
+
+    $Properties = @(
+      "UserPrincipalName",
+      "DisplayName",
+      "AuthenticationPolicy",
+      "AccountDisabled",
+      "Guid",
+      "SID"
+    )
+  }
+  Process {
+
+    # Find users without the default authentication policy
+    $NonDefaultAuthPolicyUsers = $Users | Where-Object { [string]::IsNullOrEmpty($_.AuthenticationPolicy) }
+
+    # Filter out only select properties
+    $NonDefaultAuthPolicyUsers = $NonDefaultAuthPolicyUsers | Select-Object -Property $Properties
+
+    if ($NonDefaultAuthPolicyUsers.Count) {
+      # Create an output directory and export as CSV
+      New-Item -ItemType Directory -Force -Path $OCCAM:OutputDir | Out-Null
+      $NonDefaultAuthPolicyUsers | ConvertTo-Csv -NoTypeInformation | Out-File ('{0}/users.csv' -f $OCCAM:OutputDir) -Force
+    }
+
+
+    $output = @{}
+
+    return $output
+  }
+  End {
+
+  }
+}

occam/occam.psd1

@@ -69,7 +69,8 @@
   # FormatsToProcess = @()
 
   # Modules to import as nested modules of the module specified in RootModule/ModuleToProcess
-  NestedModules = 'Internal/Build-RuleSet.ps1',
+  NestedModules = 'Internal/Build-MsolProxy.ps1',
+                  'Internal/Build-RuleSet.ps1',
                   'Internal/Invoke-TenantAudit.ps1',
                   'Internal/Invoke-TenantListGUI.ps1',
                   'Internal/Write-PSObject.ps1',