Manage network access to a HDInsight Cluster

Azure HDInsight is an Apache Hadoop distribution powered by the cloud. This means that it handles any amount of data, scaling from terabytes to petabytes on demand. Spin up any number of nodes at any time.

Since HDInsight is a PaaS offering, it is by default publicly accessable from any internet connection. The cluster contains often valuable data of customers. These customers also have requirements how to securely connect to this data, for example using IP restrictions so only their block of IP addresses can connect to the cluster.

In this article we are going to secure the HDInsight cluster so only IP adresses that we specify can connect to it.

  1. Log in to Azure using http://portal.azure.com
  2. You must have a Virtual Network (vNet) to continue, if you don’t have a vNet yet, create one.  This is mandatory.
  3. Click on +New
    1
  4. Search for HDInsight
    image
  5. Select the HDInsight Cluster
    image
  6. Click on Create

    image
  7. Give the HDInsight Cluster a name
    image
  8. Select the correct Cluster Type and Version

    image

  9. Enter the correct credentials
    image
  10. Give the Storage Account and the Container a name
    image
  11. Select the correct sizing of your cluster.
    Be aware that there is a default quota of 60 cores for a Subscription. This can be increased by raising a Support Request.
    See https://azure.microsoft.com/en-us/blog/azure-limits-quotas-increase-requests/ for more information about quotas.
    image
  12. Click on Optional Configuration and select Virtual Network
  13. Select the correct vNet:
    image
  14. Select the correct Subscription
  15. Click on Create and wait 30 minutes:

    image

  16. Now that the HDInsight Cluster is created it is accessible from the public internet. This is something many customers want to prevent, so we need to secure it.
    Since HDInsight is connected to a Private Network, we can assign a Network Security Group (NSG) and then create Inbound Security Rules to allow (not deny) traffic.

    Microsoft requires access from some IP adresses for managebility.  They provide a PowerShell script to create the Network Security Group and give these addresses access to access the cluster. This script can be downloaded here.
    The adjusted script for the environment above, can be seen here.

  17. It is necessary to modify the script and run it. It will create the Network Security Group and have the Microsoft address as inbound rules.
    Note: you cannot set Outbound Security Rules  on the Network Security Group.
    image
  18. Add your own public address, like your datacenter, home IP or office WiFi ip addresses as Inbound Security Rule
    image

 

Now the HDInsight cluster is only available from the addresses and ports that you specified in the Inbound Security Rules.

Posted in Cloud | Tagged , , , , , , , | Leave a comment

TFS / NuGet error–Missing package

Error:

S:\Program Files\Microsoft Team Foundation Server 12.0\Tools\nuget.exe restore “S:\Build\8\<name>\<name>\src\<name>.sln” -NonInteractive

WARNING: Unable to connect to the remote server

WARNING: Unable to connect to the remote server

Unable to find version ‘1.2.2’ of package ‘elmah’.

Unable to find version ‘1.2.2’ of package ‘elmah.corelibrary’

 

Solution:

0. Connect to the internet

1. Open Visual Studio

2. Open the Project

3. Open Tools, NuGet Package Manager, Package Manager Console.

4. Enter: Get-Package -ListAvailable -Filter <name of package>
for example Get-Package -ListAvailable -Filter elmah

 

5. Enter Install-Package <name>
For example: Install-Package elmah

Repeat for each missing package.

6. In Source Control Explorer, right click on the Packages folder of the Project.

7. Click on Add Items to Folder, add the new folders in the local Packages folder.

8. In Source Control Explorer, right click on the Packages folder of the Project.

9. Click on Check in Pending Changes

 

Rebuild the solution.

Posted in Microsoft General | Tagged , , , , , | Leave a comment

Error presenting PowerPoint file in Skype for Business

“<filename>.pptx couldn’t be converted to presentation because Visual Basic for Application (VBA) is not installed on this computer. Please install VBA and try again”

1. Open Add or Remove Programs in Control Panel, select Microsoft Office (Professional Plus) 2013 client and click Change

2. Select Add or Remove Features and click on Continue

image

3. Expand Office Shared Features and check if Visual Basic for Application has been installed.

image

4. Click on Continue

5. Restart Skype for Business and re-try  to present the PowerPoint file.

Posted in Microsoft General | Tagged , | Leave a comment

SQL error “Property Owner is not available for Database

SNAGHTML1e290348

Run the following command to set a new owner, in this example DBO. The database name in this example is newtraders.

use NEWTRADERS

go

sp_changedbowner @loginame = ‘dbo’

go

Posted in Microsoft General | Tagged , , | Leave a comment

SQL Server Scheduled Job fails to execute

SQL Server Scheduled Job ‘MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb’ (0x9550CE8A82FC08489D1CCDE01A38C057) – Status: Failed – Invoked on: 2016-01-25 15:16:00 – Message: The job failed.  Unable to determine if the owner (DOMAIN\ACCOUNT) of job MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb has server access (reason: Could not obtain information about Windows NT group/user DOMAIN\SAACCOUNT, error code 0x5. [SQLSTATE 42000] (Error 15404)).

This is due to the fact that the SQL Server Agent is running under a local account that cannot read the domain account permissions. Change the job to a local account or change the SQL Server Agent Service to a domain account.

Posted in Microsoft General | Tagged , , | Leave a comment