The notes below are complimentary to the youtube video showing students how to setup Jenkins on the cloud along with

• Setting up Gradle to run builds on Jenkins
• Setting up Junit to generate a test report
• Setting up Jacoco to generate a code coverage report
• Excluding classes from the Jacoco code coverage report
• Setting up a Webhook from Github to Jenkins
• Setting up a jar file as output of a jenkins build
• Configuring Gradle to wait for user input

The Video

https://www.youtube.com/watch?v=fCnmcsc7VEc

On your browser

1. Create an account on Vultr
2. Head to Products > Plus button > Deploy new server  then choose Server - Cloud compute , Server location - Sydney, Server type - Marketplace  Apps > Docker > Ubuntu 20.04 x64  
3. Head to Products > Server > Server Details and note down ip address,  username and password It should look something like 107.191.57.92 root and v3j4by@341 respectively

On your terminal

• Remote into your newly created instance via ssh

ssh <username>@<ip address>  

• Create a docker container with Jenkins official image and give it a custom name myjenkins  , do this by executing the command

docker run -d --restart always -p 80:8080 -p 50000:50000 -v jenkins_home:/var/jenkins_home --name myjenkins jenkins/jenkins:lts-jdk11

• Find your initial password by executing the command below, it should look something like a1234b32b32412asdf123vbdfgasfqwer

docker exec myjenkins cat /var/jenkins_home/secrets/initialAdminPassword

On your browser

1. Open your browser and head to <ip address> e.g. http://107.191.57.92
2. It should ask for your initial password that you copied in the previous steps  Choose Install Suggested Plugins (takes around 10mins) Create a new username and password for web access to your newly created Jenkins instance Share the ip address together with your new username and password with your team members so they can see faillure and success of builds  
3. Adding support for Jacoco  Head to Jenkins > Manage Jenkins> Manage Plugins > Available > and Type 'Jacoco', select the checkbox then click Install without restart Click Restart Jenkins when installation is complete and no jobs are running  
4. Adding support for Gradle  Head to Manage Jenkins > Global Tool Configuration > Gradle > Add Gradle > Type any name e.g. mygradle691  
5. Adding support for Github Enterprise  Head to Manage Jenkins > Configure System > GitHub Enterprise Servers > Set API Endpoint to  https://github.sydney.edu.au/api/v3 Set Name to Github USYD  

Setting up the webhook from Github to Jenkins

• Repo  https://github.sydney.edu.au/FRFU8670/fruitshop.git
• Payload URL - http://<MY IP ADDRESS>/github-webhook/
• Content type : application/json
• Disable SSL verification

FAQ

How can I ask Jenkins to pull down my source code from University of Sydney’s Github?

• Ensure that you have configured Github Enterprise in step 5 of “on your browser”
• Jenkins > Your project > Configure > Source Code Management  Choose Git Repository URL - https://github.sydney.edu.au/FRFU8670/fruitshop.git Setup your credentials Credentials  Click on Add > Jenkin and fill in your  Username : frfu8670 (your unikey) Password  : xxxxxxxx (your unikey password) Click on Add   Click on the dropdown menu and choose your newly added credentials e.g. frfu8670/*****    

How can I ask Jenkins to run the tasks to build and test my project?

• Ensure you have gradle configured properly for Jenkins - see step 4  in “On your browser”
• Head over to Jenkins > Your project > Configure > Build > Invoke Gradle script > Invoke Gradle > Gradle version and select the gradle that you configured in step 4 of “On your browser” e.g. mygradle691
• Then under Tasks section, type in clean build test which will run 3 separate tasks that will clean your project, build your project and run unit tests on said project

How can I add a report to show my unit test results?

• Head over to Jenkins > Your project > Configure > Post-build Actions > Publish Junit test result report  and find the section that says Test report XMLs and enter the following and press Save

**/test-results/**/*.xml

How can I add a report to show my unit test coverage?

• Ensure you have the Jacoco plugin installed for Jenkins - see step 3 in  “On your browser”
• Head over to Jenkins > Your project > Configure > Post-build Actions > Record JaCoCo coverage report and press Save

How can I exclude a class from being included in the Test Coverage Report?

• Head over to Jenkins > Your project > Configure > Post-build Actions > Record JaCoCo coverage report and add the following under Exclusions. For example if you had a class called App.java and which had non backend business logic such as asking for command line user input and cleaning and you wish to exclude in your test coverage report  you can enter the following in the input field undernearth the Exclusions and press Save

**/App.class

How can I ask Github to notify Jenkins when a new commit has been pushed?

• Head over to Jenkins > Your project > Configure > Build Triggers  Choose GitHub hook trigger for GITScm polling and press Save  
• Head over to Github > Your repository > Settings > Hooks > Add webhook  Payload URL - http://<your server ip>/github-webhook/ e.g. http://107.191.57.92/github-webhook/ then choose Content type - application/json , SSL Verification - Disable then click on Add webhook  

How can I ask Jenkins to find out if a new commit has been pushed without Github notifying it?

• Head over to Jenkins > Your project > Configure > Build Triggers  Choose Poll SCM Type in H/1 * * * * underneath Schedule  
• This will ask Jenkins to poll your repository every 1 minute for changes such as new  commit pushed and merged pull requests etc.

How can I ask Jenkins to build a jar file that contains my command line app?

• Head over to project code and add  a jar task in your app/build.gradle file jar

jar {
    manifest {
        attributes(
            "Main-Class": 'fruitshop.App'
        )
    }
}

• Then head over to Jenkins > Your project > Configure > Add post-build action  Choose Archive the artifacts Under Files to archive type in app/build/libs/app.jar or whatever the name of your app is called  

How can I ask Gradle to wait for user input?

• Head over to your app/build.gradle file and give some configuration options to the run task

run {
   standardInput = System.in
}

• Then every time you would like to run your application instead of gradle run you would execute

gradle run --console=plain

warning: agent returned different signature type ssh-rsa (expected rsa-sha2-512)

After performing the obligatory google search, I stumbled across several threads but still couldn't find a solution. This annoyed me enough to look for other alternatives and discovered a Windows port of OpenSSH Portable. So since I didn't want (probably not even possible) to have both binaries (and ssh-agents) running side by side. I decided it would be best that I remove Window's 1809's installed version first then install OpenSSH Portable

Uninstall Windows' OpenSSH

This basically removes the binaries installed in C:\Windows\System32\OpenSSH

Remove-WindowsCapability -Online -Name "OpenSSH.Client~~~~0.0.1.0"
Remove-WindowsCapability -Online -Name "OpenSSH.Server~~~~0.0.1.0"

Stop and delete the existing Windows Service

Get-Service ssh-agent | Stop-Service
sc.exe delete ssh-agent

Install OpenSSH Portable

I'm using Chocolatey's openssh package but you can download and install the binaries yourself here. This installs the binaries into C:\Program Files\OpenSSH-Win64 . The /SSHAgentFeature flag ensures that the SSH Agent Service gets installed too.

choco install openssh --package-parameters="/SSHAgentFeature"

Note version 7.9.0.1 was the last good working version for me. 8.0.0.1 was complaining about my ssh key being in an invalid format.

Check that you have the binaries installed

Get-Command ssh*exe
    
    CommandType Name              Version Source
    ----------- ----              ------- ------
    Application ssh.exe           7.9.0.0 C:\Program Files\OpenSSH-Win64\ssh.exe
    Application ssh-add.exe       7.9.0.0 C:\Program Files\OpenSSH-Win64\ssh-add.exe
    Application ssh-agent.exe     7.9.0.0 C:\Program Files\OpenSSH-Win64\ssh-agent.exe
    Application sshd.exe          7.9.0.0 C:\Program Files\OpenSSH-Win64\sshd.exe
    Application ssh-keygen.exe    7.9.0.0 C:\Program Files\OpenSSH-Win64\ssh-keygen.exe
    Application ssh-keyscan.exe   7.9.0.0 C:\Program Files\OpenSSH-Win64\ssh-keyscan.exe
    Application ssh-shellhost.exe 7.9.0.0 C:\Program Files\OpenSSH-Win64\ssh-shellhost.exe

Check that you have the ssh-agent running

Get-Service ssh-agent

Status   Name               DisplayName
------   ----               -----------
Running  ssh-agent          ssh-agent

Configure our git client to use our newly installed binaries for ssh

git config --global core.sshCommand "'C:\Program Files\OpenSSH-Win64\ssh.exe'"

Ensure that it has been configured properly

Get-Content $env:USERPROFILE\.gitconfig | Select-String sshCommand -Context 6

[core]
        excludesfile = ~/.gitignore_global
        sshCommand = 'C:\\Program Files\\OpenSSH-Win64\\ssh.exe'

I wanted to remove any existing identities (but this could be optional)

ssh-add -D

Add new identites

ssh-add 

Then we should be good to go ;)

We are currently using AWS Cloudfront to front our backend .NET web application (the origin). A new requirement came through asking us to serve up customised content based on whether the originating request was from a bot or human.

By default, AWS Cloudfront does not forward the user request's User-Agent to the origin so what can we do? Well one (non-peformant) way is to simply modify our distribution behavior to whitelist the the User-Agent header but this puts a punch to our hit/miss ratio because different users with different browser versions and device types will generated a bunch of different user agents (e.g. Safari on MacOS, Opera on Smart TV, Chrome on iPad etc).

The solution

So how can we let our origin serve up customised content for bots yet stay (somewhat) performant? Lambda@Edge to the rescue! Lambda@Edge lets you run Lambda functions to customize content that CloudFront delivers by executing the functions at edge locations. You can modify the request/response during any of the following stages:

• After CloudFront receives a request from a viewer (viewer request)
• Before CloudFront forwards the request to the origin (origin request)
• After CloudFront receives the response from the origin (origin response)
• Before CloudFront forwards the response to the viewer (viewer response)

My proposed solution still requires us to whitelist the User-Agent header but additionally modify the viewer request by setting the User-Agent to Human if it doesn't match any whitelisted bot. If a whitelisted bot is matched then keep the bot's User-agent and forward it to our origin for further content customisation. This way, real humans will always have a User-Agent of Human and not the device/browser/version variant (e.g. Safari on MacOS, Opera on Smart TV, Chrome on iPad etc) while being able to identify the User-Agent of the whitelisted bot at the same time. I've created the the Lambda function below which is deployed to my Cloudfront Distribution.

Step 1

Whitelist the User-Agent header in our distributions behavior over on our AWS console (or AWS CLI)

CloudFront Distributions > E2VZ0ISAMPLEDST > Behaviors > Edit > Whitelist Headers > Enter a custom header "User-Agent" > Add Custom > Yes, Edit

Step 2

Create and deploy our Lambda function. If you are new to Lambda functions like I was, simply follow their tutorial to creating a simple Lambda@Edge function. Once you are comfortable creating a Lambda function, you can create your own. Here is mine.

Lambda Function.  Triggered when CloudFront receives a request from a viewer (viewer request)

'use strict';
 
exports.handler = (event, context, callback) => {
    const request = event.Records[0].cf.request;
    const headers = request.headers;
    const uri = request['uri'];
    const botPattern = "Googlebot\\/|Googlebot-Mobile|Googlebot-Image|Googlebot-News|Googlebot-Video|bingbot";
    var re = new RegExp(botPattern, 'i');
    var userAgent = headers['user-agent'][0]['value'];

    if (re.test(userAgent)) {
        headers['user-agent'] = [{key: 'User-Agent', value: userAgent}];
    } else {
        headers['user-agent'] = [{key: 'User-Agent', value: 'Human'}];
    }

    callback(null, request);
};

Now obviously the botPattern isn't an exhaustive list because it's my personal whitelist. You can implement your own logic however you like ;)

If anyone has suggestions to improve it, please leave your comments below!

References

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-how-to-choose-event.html

If you have an existing VM, you can enable it by

Set-VMFirmware TestVM -SecureBootTemplate MicrosoftUEFICertificateAuthority

If you want to start from scratch you can follow the script below, it assumes that you've already created a virtual switch and have downloaded a copy of Ubuntu Server

Create and configure

# VM creation
$vmName = "UBUSRV";
$vmNewDiskPath = "F:\HyperV\Virtual Hard Disks\UBUSRV.vhdx";
$vmNewDiskSize = 20GB;
$vmPath = "F:\HyperV\Virtual Machines";
$vmGeneration = 2;
$vmBootDevice = "VHD";
$vmSwitchName = "MyVirtualSwitch"; # To find existing switches run, Get-VMSwitch | ft
$vmDvdDrivePath = "C:\Users\Frank\Downloads\ubuntu-18.04.1.0-live-server-amd64.iso"

$vmFirmwareEnableSecureBoot = "On"; # Turn off if you trust and/or image isn't supported.
$vmFirmwareSecureBootTemplate = "MicrosoftUEFICertificateAuthority";

$vmProcessorCount = 4;
$vmMemoryStartUpBytes = 1GB;
$vmMemoryMinimumBytes =  500MB;
$vmMemoryMaximumBytes =  3GB;
$vmDynamicMemoryEnabled = $true;

New-VM -Name $vmName -BootDevice $vmBootDevice -NewVHDPath $vmNewDiskPath -Path $vmPath -NewVHDSizeBytes $vmNewDiskSize -Generation $vmGeneration -SwitchName $vmSwitchName
Set-VMFirmware $vmName -EnableSecureBoot $vmFirmwareEnableSecureBoot -SecureBootTemplate $vmFirmwareSecureBootTemplate
Set-VMProcessor $vmName -Count $vmProcessorCount
Set-VMMemory $vmName -DynamicMemoryEnabled $vmDynamicMemoryEnabled -MinimumBytes $vmMemoryMinimumBytes -StartupBytes $vmMemoryStartUpBytes -MaximumBytes $vmMemoryMaximumBytes
Add-VMDvdDrive $vmName -Path $vmDvdDrivePath # To eject run Remove-VMDvdDrive $vmName

Debug and clean up

# Debugging
Get-VMFirmware $vmName | fl
Get-VMProcessor $vmName | fl
Get-VMMemory $vmName | fl
Get-VMDVDDrive $vmName | fl 

# Clean up
Remove-VM -Name $vmName -Force
Remove-Item $vmNewDiskPath -Force

Related links

https://www.ubuntu.com/download/server

https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/supported-linux-and-freebsd-virtual-machines-for-hyper-v-on-windows
https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/supported-ubuntu-virtual-machines-on-hyper-v
https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/what-s-new-in-hyper-v-on-windows#BKMK_linux

• Spin up a Windows server with GUI in Hong Kong or Japan then just remote deskop into the instance.
• Spin up a Linux server in Hong Kong or Japan and setup a VPN and connect to this VPN using a Connect VPN client.
• Purchase a VPN that China doesn't actively block. PIA for example gets actively blocked by the great firewall where as a service like ExpressVPN works.
• Purchase a data sim from outside China which is usable in China. Sims Direct , an Australian company, provides an data only sim which lets you use it in 18 asian countries for a span of 15 days

I personally prefer the second method, because this means I can then download Open VPN Client on all my devices (iPad, iPhone, Laptop etc).

Spinning up a Linux box with OpenVPN Access Server either pre-installed or self installation, configure the wizard, add desired users and permissions then lastly login as the desired user and download the  Connection Profile in the .ovpn format by logging in https://<openvpnasip>:943 and clicking the links in "Connection profiles can be downloaded for" section. You might have open the .ovpn file and find and replace any occurance of local to public IP addresses.

A quick scan of Get-Service ssh-agent | select * showed the following

UserName            : LocalSystem
Description         : Agent to hold private keys used for public key authentication.
DelayedAutoStart    : False
BinaryPathName      : C:\Windows\System32\OpenSSH\ssh-agent.exe
StartupType         : Disabled
Name                : ssh-agent
RequiredServices    : {}
CanPauseAndContinue : False
CanShutdown         : False
CanStop             : False
DisplayName         : OpenSSH Authentication Agent
DependentServices   : {}
MachineName         : .
ServiceName         : ssh-agent
ServicesDependedOn  : {}
StartType           : Disabled
ServiceHandle       : Microsoft.Win32.SafeHandles.SafeServiceHandle
Status              : Stopped
ServiceType         : Win32OwnProcess
Site                :
Container           :

I quickly noticed that the StartType was set to Disabled. Not sure for what reason?  To fix this I had to first change it back to Manual then start the service again.

Get-Service -Name ssh-agent | Set-Service -StartupType Manual
Start-Service ssh-agent

But it didn't end here, I also had to instruct git to look for ssh in Windows' directory

git config --global core.sshCommand C:/Windows/System32/OpenSSH/ssh.exe

So now that git knows where to look for ssh we can now add our private key to the ssh authentiction agent ssh-add then double checking with ssh-add -l

2048 SHA256:p80/dPfeSzXZPM7ba61214oXCNzMB+v+s/K8gexampleaWzx7Y /home/owo/.ssh/id_rsa (RSA)

On the Azure Portal head over to your app service then jump over to the Kudu > Debug console > CMD and create a file name applicationHost.xdt under d:\home\site directory which allows you set the allowed server variables for IIS by overriding/transforming the applicationHost.config file

applicationHost.xdt

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <system.webServer>
    <rewrite>
      <allowedServerVariables xdt:Transform="Replace">
        <add name="HTTP_X_ORIGINAL_HOST" xdt:Transform="InsertIfMissing" />
        <add name="HTTP_HOST" xdt:Transform="InsertIfMissing" />
      </allowedServerVariables>
    </rewrite>
  </system.webServer>
</configuration>

Then on Web.config file within d:\home\site\wwwroot you'll need a new rule which basically sets the HTTP_HOST server variable to be {HTTP_X_ORIGINAL_HOST} (surrounding curly brace means to evaluate) on the condition that the value of the http header X-Original-Host is NOT empty (e.g. ^$).

X-Original-Host is a HTTP header which Azure's Application Gateway forwards to its backend pools

Web.config

<configuration> 
  <system.webServer>
    <rewrite>
      <rules>
        <rule name="Set HOST header based on X-Original-Host header">
          <match url="(.*)"></match>
          <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
            <add input="{HTTP_X_ORIGINAL_HOST}" pattern="^$" negate="true" />
          </conditions>
          <serverVariables>
              <set name="HTTP_HOST" value="{HTTP_X_ORIGINAL_HOST}"></set>
          </serverVariables>
        </rule>
      </rules>      
     </rewrite>
  </system.webServer>
</configuration>

IMPORTANT : Once the two changes are made, we will need to restart BOTH the SCM site (via the Restart Site button on {foo}.scm.azurewebsites.net/SiteExtensions) and also the website itself (Via the Azure Portal, CLI or Powershell) or the changes won't kick in.

And to test whether the correct headers are being transformed, simply add some debugging code in your html (or razor page in my case of ASP.NET MVC)

Index.cshtml

<div class="jumbotron">    
    <p>HTTP_HOST : @Request.ServerVariables["HTTP_HOST"]</p>
    <p>HTTP_X_ORIGINAL_HOST : @Request.ServerVariables["HTTP_X_ORIGINAL_HOST"]</p>   
</div>

<div class="jumbotron">
    <p>X-Forwarded-For : @Request.Headers["X-Forwarded-For"]</p>    
    <p>X-Forwarded-Proto : @Request.Headers["X-Forwarded-Proto"]</p>
    <p>X-Original-Host : @Request.Headers["X-Original-Host"]</p>    
 </div>

Side note for deployments outside of App Service

If we have greater control over the hosting environment, perhaps using App Service for Windows Containers (in Public preview at time of writing) the same could be achieved using powershell with Set-WebConfigurationProperty cmdlet in your Dockerfile while building your image. The below example assumes you've built your image using the Default Web Site but can obviously be changed to however your website is configured.

Set-WebConfigurationProperty -PSPath 'MACHINE/WEBROOT/APPHOST' -location 'Default Web Site' ` 
-filter 'system.webServer/rewrite/allowedServerVariables' -Name '.'  `
-value ( @{ name = 'HTTP_HOST' }, @{ name = 'HTTP_X_ORIGINAL_HOST' } )

Which would produce

applicationHost.config

<configuration>
    <location path="Default Web Site" overrideMode="Allow">
            <system.webServer>
                <rewrite>
                    <allowedServerVariables>
                        <add name="HTTP_X_ORIGINAL_HOST" />
                        <add name="HTTP_HOST"/>
                    </allowedServerVariables>
                </rewrite>
                <asp />
            </system.webServer>
    </location>
</configuration>

Here is the Dockerfile for completeness

Dockerfile

ARG LTSC_VERSION=ltsc2016
FROM mcr.microsoft.com/dotnet/framework/aspnet:4.7.2-windowsservercore-${LTSC_VERSION}

RUN iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
RUN choco install urlrewrite -y

WORKDIR /inetpub/wwwroot 

RUN Set-WebConfigurationProperty -PSPath 'MACHINE/WEBROOT/APPHOST' -location 'Default Web Site' -filter 'system.webServer/rewrite/allowedServerVariables' -Name '.'  -value ( @{ name = 'HTTP_HOST' }, @{ name = 'HTTP_X_ORIGINAL_HOST' } )

ARG PUBLISH_DIR
ADD output/${PUBLISH_DIR}/. ./

References

https://github.com/projectkudu/kudu/wiki/Azure-Site-Extensions#understanding-what-could-go-wrong-with-xdt-transforms

# Setup variables
$companyName = "contoso"
$certFilePath = "$PSScriptRoot\$companyName-wildcard.pfx"
$dnsName = "*.$companyName.com"
$date = (Get-Date).ToString('MMM-yyyy')
$certFriendlyname = "$companyName-wildcard-$date"
$certExpiry = (Get-Date).AddYears(10)

# Removal existing certificates
gci Cert:\LocalMachine\Root | Where FriendlyName -Like "*$companyName*" | Remove-Item
gci Cert:\LocalMachine\My | Where FriendlyName -Like "*$companyName*" | Remove-Item

# Create self signed certificate and store thumbprint variable
$thumb = (New-SelfSignedCertificate -DnsName $dnsName -CertStoreLocation cert:\LocalMachine\My -FriendlyName $certFriendlyname -NotAfter $certExpiry).Thumbprint

# Export self signed certificate to local file system with mandatory password
$pwd = ConvertTo-SecureString -String "YourPassword" -Force -AsPlainText
Export-PfxCertificate -Cert cert:\LocalMachine\My\$thumb -FilePath $certFilePath -Password $pwd

# Adding the exported certificate into the Trusted Root Certificate Authorities store
Import-PfxCertificate -CertStoreLocation Cert:\LocalMachine\Root\ -FilePath $certFilePath -Password $pwd

# Rebinding existing ssl certificates in case Windows decides to remove third party certificates from Root
$bindings = Get-WebBinding | Where { $_.Protocol -eq "https" -and $_.bindingInformation -like "*$companyName*" }
$cert = gci Cert:\LocalMachine\MY | Where Subject -Like *$companyName* | select -First 1
foreach ($b in $bindings)
{
    $b.RebindSslCertificate($cert.GetCertHashString(), "My") | Out-Null
}

We can verify that all this has been done correctly by heading over to Windows > run > certlm.msc

• Personal > Certificates and
• Personal > Trusted Root Certification

And then check for the entry with "Issued By" as *.contoso.com

Then we will need to add an entry to override DNS by editing our host file on
c:\Windows\System32\Drivers\etc\hosts.

127.0.0.1     test.contoso.com

Once all this is done, we can fire up IIS Manager (inetmgr) and add a binding to the website of our choice with the newly generated certificate.

Check https://test.contoso.com on your favourite browser and make sure the location bar is green.

Reversal for everything above

Remove-Item Cert:\LocalMachine\My\$thumb
Remove-Item Cert:\LocalMachine\AuthRoot\$thumb
Remove-Item C:\contoso-wildcard.pfx

Important notes

It appears that sometimes these certificates get removed from the Root Store which results in errors like

This server could not prove that it is test.contoso.com; its security certificate is not trusted by your computer's operating system. This may be caused by a misconfiguration or an attacker intercepting your connection.

It turns out that Windows has the ability to remove certs. Still need to figure out why and how we can stop this from happening.

These links could be related

https://serverfault.com/questions/639280/trusted-root-certificate-being-automatically-removed-from-store
https://serverfault.com/questions/752146/why-are-many-admins-using-turn-off-automatic-root-certificates-update-policy
https://serverfault.com/questions/526736/preventing-windows-from-deleteing-our-root-ca
https://superuser.com/questions/217719/what-are-the-windows-system-certificate-stores

NiceHash Miner is an incredibly simple program to use and I've been wanting to start and stop the program using the Windows Task Scheduler. I initially tried stop the task by using the "Stop task if running more than x" option in the settings section but quickly found that NiceHash Miner 2.exe actually launches another process named excavator.exe (and not as a child process) which doesn't get stopped even though NiceHash Miner 2.exe was stopped by the Windows Task Scheduler.

The work around was to create a helper in the form of a powershell script to start and stop the individual processes. You'll see the end result below.

Note : You'll also need to enable the "Autostart mining" within NiceHash Miner for this to work.

Set execution policy to remote signed

Set-ExecutionPolicy RemoteSigned

Creating a powershell script to start/stop NiceHash Miner processes

NiceHashHelper.ps1

# usage : e.g. "powershell .\NiceHashHelper.ps1 -action start" 

param ([string]$action = "Start")

if($action -eq "Start")
{
    Write-Host "Starting.."
    Start-Process "$env:LOCALAPPDATA\Programs\NiceHash Miner 2\NiceHash Miner 2.exe"
} 
elseif ($action -eq "Stop")
{
    Write-Host "Stopping.."
    Stop-Process -Name *NiceHash*            # The main program
    Stop-Process -Name *excavator*           # The GPU miner
    Stop-Process -Name *xmr-stak-cpu*        # The CPU miner
}
else {
    Write-Host "Invalid action : Please choose either Start or Stop with the action param"
}

Creating a task in Task Scheduler

# Variables
$cmd = "$env:USERPROFILE\Desktop\NiceHash\NiceHashHelper.ps1"
$argumentPrefix =  '-command "& {0} -action "' -f ($cmd)
$startTrigger =  New-ScheduledTaskTrigger -Daily -At 6pm
$stopTrigger =  New-ScheduledTaskTrigger -Daily -At 7am
$taskFolderName = '\NiceHash Tasks'
$executable = "Powershell.exe"

# The "Start" task
$startActionArgument = '{0} {1}' -f ($argumentPrefix, "Start")
$startAction = New-ScheduledTaskAction -Execute $executable -Argument $startActionArgument
Register-ScheduledTask -Action $startAction -Trigger $startTrigger -TaskName "Start NiceHash Miner" -TaskPath $taskFolderName

# The "Stop" task
$stopActionArgument = '{0} {1}' -f ($argumentPrefix, "Stop")
$stopAction = New-ScheduledTaskAction -Execute $executable -Argument $stopActionArgument
Register-ScheduledTask -Action $stopAction -Trigger $stopTrigger -TaskName "Stop NiceHash Miner" -TaskPath $taskFolderName

# Unregister-ScheduledTask -TaskName "Start NiceHash Miner"
# Unregister-ScheduledTask -TaskName "Stop NiceHash Miner"

Checking for irregularities

use master

-- check for recent events
select top 1000 * from sys.event_log order by start_time desc

-- filter by severity
select top 100 * from sys.event_log where severity > 0 order by start_time desc

To view more details about the dead locks

use master

WITH CTE AS (

       SELECT CAST(event_data AS XML)  AS [target_data_XML]

       FROM sys.fn_xe_telemetry_blob_target_read_file('dl', null, null, null)

)

SELECT target_data_XML.value('(/event/@timestamp)[1]', 'DateTime2') AS Timestamp,

target_data_XML.query('/event/data[@name=''xml_report'']/value/deadlock') AS deadlock_xml,

target_data_XML.query('/event/data[@name=''database_name'']/value').value('(/value)[1]', 'nvarchar(100)') AS db_name

FROM CTE

References

https://blogs.msdn.microsoft.com/azuresqldbsupport/2017/01/21/lesson-learned-19-how-to-obtain-the-deadlocks-of-your-azure-sql-database/

Import-Module WebAdministration
foreach($WebSite in $(get-website))
{
    $logFile="$($Website.logFile.directory)\w3scv$($website.id)".replace("%SystemDrive%",$env:SystemDrive)
    Write-host "$($WebSite.name) [$logfile]"
}

# Results
mysite1.dev[C:\inetpub\logs\LogFiles\w3scv2]
mysite2.dev[C:\inetpub\logs\LogFiles\w3scv3]
mysite3.dev[C:\inetpub\logs\LogFiles\w3scv4]
mysite4.dev[C:\inetpub\logs\LogFiles\w3scv5]
mysite5.dev[C:\inetpub\logs\LogFiles\w3scv1]
mysite6.dev[C:\inetpub\logs\LogFiles\w3scv6]

# Let's tail the last 5 rows mysite1's log and wait for more
gc C:\inetpub\logs\LogFiles\w3scv2\u_ex180316.log -tail 5 -wait

Make sure WinRM service is running and start up to automatic

Get-Service winrm

Enable Powershell Remoting

Enable-PSRemoting -Force

Ensure network profiles are ok

# Show network profile
> Get-NetConnectionProfile

Name             : Network
InterfaceAlias   : Ethernet
InterfaceIndex   : 2
NetworkCategory  : Public
IPv4Connectivity : Internet
IPv6Connectivity : NoTraffic

# Set network profile from Public to Private
> Get-NetConnectionProfile | Set-NetConnectionProfile -NetworkCategory Private

# Verify the update completed
> Get-NetConnectionProfile

Name             : Network
InterfaceAlias   : Ethernet
InterfaceIndex   : 2
NetworkCategory  : Private
IPv4Connectivity : Internet

Recently wanted to setup a Windows Server Core VM/image for portability. It's mainly used to build ASP.NET MVC web apps on the full .NET Framework which would then need to be deploy to Azure. Just documenting the steps I took along the way.

# Download and install VS Build Tools
Invoke-WebRequest https://aka.ms/vs/15/release/vs_buildtools.exe -OutFile vs_buildtools.exe ;
Start-Process -FilePath 'vs_BuildTools.exe' -ArgumentList `
  '--quiet', '--norestart', '--locale en-US' `
, '--add Microsoft.VisualStudio.Workload.NetCoreBuildTools' `
, '--add Microsoft.VisualStudio.Workload.WebBuildTools' `
, '--includeOptional', '--includeRecommended' -Wait ;

References

Other Workloads
Installer CLI Docs
Installer CLI Examples

Setup Installers

# Install chocolatey
Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

# Choco setup
choco feature enable -n allowGlobalConfirmation

Setting up other dependencies

My builds require Web Deploy and NodeJS so let's install them using Chocolatey

choco install git 
choco install webdeploy
choco install nodejs-lts
choco install nuget.commandline

Next steps

Would like to containerize this image with exposable environment variables such as VSTS URLs, PAT etc.

Some folk have been arriving to this post hoping to find a solution for Windows Server Core might need to look elsewhere. I've only tested this on Windows Server 2012.

Service requirements

We need to make sure the following services are running

# Check status of services required for Network Discovery on Windows
Get-Service -DisplayName "Function Discovery Resource Publication"
Get-Service -DisplayName "DNS Client" 
Get-Service -DisplayName "SSDP Discovery" 
Get-Service -DisplayName "UPnP Device Host"

If not any are not running, start them

# Start required services
Get-Service -DisplayName "Function Discovery Resource Publication" | Start-Service
Get-Service -DisplayName "DNS Client"  | Start-Service
Get-Service -DisplayName "SSDP Discovery"  | Start-Service
Get-Service -DisplayName "UPnP Device Host" | Start-Service

Firewall requirements

Check whether respective firewall rules are both set to Allow and also Enabled

Get-NetFirewallRule -DisplayGroup "Network Discovery" | ft 
Get-NetFirewallRule -DisplayGroup "File and Printer Sharing" | ft

If not, allow and enable them

# Set action to allow
Get-NetFirewallRule -DisplayGroup "Network Discovery" | Set-NetFirewallRule -Action Allow
# Enabling the rule
Get-NetFirewallRule -DisplayGroup "Network Discovery" | Enable-NetFirewallRule

# Set action to allow
Get-NetFirewallRule -DisplayGroup "File and Printer Sharing" | Set-NetFirewallRule -Action Allow 
# Enabling the rule
Get-NetFirewallRule -DisplayGroup "File and Printer Sharing" | Enable-NetFirewallRule

References

https://support.microsoft.com/en-au/help/2722035/you-cannot-turn-on-network-discovery-in-network-and-sharing-center-in

I ended up having to log in and out of Google Chrome, Firefox, Edge etc just to test this so I created a script to do so instead.

$loginUrl = "https://example.com/log-in"
$normalUrl = "https://example.com/logged-in-area"

$postParams = @{email='foo@bar.com';password='1234'}
iwr -Uri $url  -Method POST -Body $postParams -SessionVariable MySessionVariable
iwr $normalUrl -WebSession $MySessionVariable

# Optionally view what cookies got stored
$MySessionVariable.Cookies.GetCookies($url)

The magic lies in the MySessionVariable variable which is basically a Cookie object that gets passed in each invocation.