How to configure SQL server Fail over clustering Instance (FCI).

This blog is part of a SQL HA-DR solution series , In my previous blogs I mentioned how Log Shipping, Mirroring & AlwaysOn Availability Group can be configured, now here you will get step by step procedure for SQL Fail Over Cluster Instance (FCI) high Availability solution. SQL FCI is sometime also known as AlwaysOn FCI and it’s bit different then AlwaysOn AG (Availability group). Always ON FCI need shared storage that is accessible from all the participant node and it provide instance level high availability.  If your primary (or active) server is down then secondary (passive) take responsibilities for all SQL operation.

The details about SQL FCI can be found here.

Continue reading

Advertisements

Unable to re-create Availability group with same name

While working in my test environment, we have delete existing Availability group and try creating a new one with same name. Unfortunately It was failing with error  “Failed to create availability group ‘SQL AAG’, because a Windows Server Failover Cluster (WSFC) group with the specified name already exists.”

erroraag

We successfully deleted existing configuration as per Microsoft & Fail-over cluster log does not give any clue about existing AAG group.

Here is the solution.

  1. Drop the availability group if not in previous attempt, refer
  2. Take backup of registry then delete key “HKEY_LOCAL_MACHINE\Cluster \HadrAgNameToldMap” from all nodes participating in cluster.
  3. From SQL configuration manager, uncheck ‘Enable Alwayson Availability Groups’, apply OK. Restart SQL service.config
  4. Check the box ‘Enable Alwayson Availability Groups’ again, Apply-ok, restart SQL service.

Now you should be able to create availability group with same existing name.

successaag

#aag, #alwayson

How to safely decommission spare Exchange 2016 Server from organization.

There are many occasion when an exchange admin needs to decommission an un-used Exchange server from organization that might be installed by mistake and doesn’t full fill business requirement. There are lots of blogs or link present on internet that may guide you to do via ADSI Edit but the preferred way is to do gracefully using front end (exchange PowerShell OR GUI). Following PS script is verified in Exchange 2016 test environment for the same purpose.

$Exch2k16="exmbx2" #Exchange Server name that needs to be removed. 
$DestDb="Mailbox Database 1529611009" #Destination mailbox Database where the mailboxes of server will be moved. 
Get-MoveRequest|Remove-MoveRequest -Confirm:$False #Delete move mailbox request if exists
Get-Mailbox -Server $Exch2k16 |New-MoveRequest -TargetDatabase $DestDb #Move mailboxes to other database. 
Get-Mailbox -AuditLog -Server $Exch2k16 |New-MoveRequest -TargetDatabase $DestDb  #Move Auditing mailbox
Get-Mailbox -Monitoring -Server $Exch2k16 |New-MoveRequest -TargetDatabase $DestDb #Move health/monitoring db
Get-Mailbox -Archive -Server $Exch2k16 |New-MoveRequest -TargetDatabase $DestDb #Move all archives
Get-Mailbox -PublicFolder -Server $Exch2k16 |New-MoveRequest -TargetDatabase $DestDb #Move Publicfolder mailboxes
Get-MailboxDatabase -Server $Exch2k16 |Remove-MailboxDatabase -Confirm:$False #Remove MailboxDB
Get-ReceiveConnector -Server $Exch2k16|Remove-ReceiveConnector -Confirm:$False #Remove Receive connector.  

Now uninstall exchange binaries using following command.

setup /m:uninstall

ExchangeUn.jpg

 

#exchange, #movemailbox

Overview of Memory-Optimized Table.

We all know, retrieving data from physical memory (AKA RAM) is faster than the data saved in physical disk (HDD). Memory-optimized tables save & retrieve data into/from physical memory rather than Hard-disk. This could be excellent feature to solve lot of use cases such as online game where only end results matters.

This feature was introduced with SQL version 2014 and continuous improvement are being done in consecutive releases. In lot of places, you would see this feature named as “Hekaton” OR “In-Memory optimization”.

Following a short demo may help to develop your understanding regarding Memory table & performance comparison with disk based table.

-- Create InMemoryTableDemo database. 
USE [master]
CREATE DATABASE [InMemoryTableDemo]
GO
-- Create Memory optimized File Group.
USE [master]
ALTER DATABASE [InMemoryTableDemo]
ADD FILEGROUP MemoryGroupFG CONTAINS MEMORY_OPTIMIZED_DATA
-- Add the container for file group. 
ALTER DATABASE [InMemoryTableDemo]
ADD FILE (NAME = AWMemFile, FILENAME = N'F:\FGData\FG1') TO FILEGROUP MemoryGroupFG
GO

In my following example, I have set ‘Durability’ to ‘schema_only’ which means only the schema will be recoverable in case of crash as the rows will remain in memory. The other option is ‘Schema_and_Data’ that protect table schema as well rows, the copy of data will also save in HDD.

-- Create Memory Based Table.
USE [InMemoryTableDemo]
CREATE TABLE TableInMemory
(
       ID INT CONSTRAINT [pk_TableInMemory_ID] PRIMARY KEY NONCLUSTERED IDENTITY (1,1), 
       EMPID INT,
       CharData VARCHAR(100)
) WITH ( MEMORY_OPTIMIZED = ON , DURABILITY = SCHEMA_ONLY )
GO

Let’s create a disk based table with same columns.

-- Create Disk Based Table.
USE [InMemoryTableDemo]
CREATE TABLE TableInDisk
(      
       ID INT CONSTRAINT [pk_TableInDisk_ID] PRIMARY KEY NONCLUSTERED IDENTITY (1,1), 
       EMPID INT, 
       CharData VARCHAR(100)
)
GO

Now we are inserting 100k rows and it takes only (approximately) 4 seconds to insert in memory based table whereas disk based table takes (approximately) 200 seconds.

-- Test 100,000 inserts into a Memory Based Table 
DECLARE @start DATETIME = GETDATE();
DECLARE @count INT = 1;
WHILE @count < 100000
BEGIN
  INSERT INTO TableInMemory VALUES 
       (@count, 'stuff');
        SET @count = @count + 1;
END
SELECT DATEDIFF(s, @start, GETDATE()) AS [Memory Insert]
GO

-- Test 100,000 inserts into a Disk Based Table
DECLARE @start DATETIME = GETDATE();
DECLARE @count INT = 1;
WHILE @count < 100000
BEGIN
  INSERT INTO TableInDisk
VALUES 
       (@count, 'stuff');
       SET @count = @count + 1;
END
SELECT DATEDIFF(s, @start, GETDATE()) AS [Memory Insert]
GO

first

Let’s test the retrieval of rows. Run following 3 queries together while enabling ‘Include actual execution plan’ OR Live query statistics‘.

-- Run following 3 queries together while enabling 'include actual execution plan' OR 'Live querystatistics' 
USE [InMemoryTableDemo]
SELECT * FROM TableInDisk
SELECT * FROM TableInMemory
GO

Comparsion.jpg

You would notice Memory table takes (approximately) 16 % resources as compare to disk based table that takes 84 % of resources. Additionally storage & I/O Cost columns will give the details of they type of query.

More details on Memory optimized table.

#hekaton, #in-memory-optimization, #sql2016, #storage

How to join VMware ESX / VCenter to Active Directory domain and manage using domain account.

Most of organization uses AD infrastructure for authentication and administrator the resources of organization. ESX servers and VSphere can also be joined to AD domain and administrator using domain account.

Here are the steps to do the same.

Confirm the appropriate DNS address & domain name is configured for ESX.  Login to ESX using VSphere client, select Configuration tab, select ‘DNS and routing’. If correct DNS IP is not configured then click on ‘Properties’ and save valid details.

1.jpg

Click on Authentication Services, select Properties to edit current settings, select directory service type as Active Directory. Type domain name then join to domain by giving domain admin credential.

2.jpg

Once ESX joined in the domain, you should be able to see computer account listed in ADUC (Active directory user’s & computer)

3.jpg

Now create a group called ‘VMWare Admin’ & a user ‘VAdmin’, this user will be the member of ‘VMware admin’ group.

4.jpg

In your VSphere client, select Permission tab, right click on empty space select ‘Add Permission’.

5.jpg

Click on ADD, from drop down menu select the domain, you should be able to see AD objects, select VMware admin group.

6.jpg

Select the role you wish to give to map with AD group ‘VMware admin’.

7.jpg

Once it added successfully, you should be able to login in ESX Vsphere client using domain credential. You can check box ‘use windows session credential’ if you wants to login using current windows login.

8.jpg

If you are managing ESX using VMWare VCenter then open VCenter page, go to ‘Administration’ page, select ‘Configuration’, In ‘Identity source’ option select ‘Active Directory (integrated windows authentication)’, Type appropriate domain name and click on OK.

9.jpg

Click on ‘Global Permission’ Tab, click on ‘+’ option then Add AD group ‘VMware admin’.

11.jpg

Select the role you wish to map with to ‘VMware admin’ AD group.

12.jpg

Now you should be able to login to VCenter using windows & AD authentication.

13.jpg

#active-directory, #vmware, #windows

How to create ISCSI shared disk in windows server for ESX VSphere environment.

This my first blog for ESX/VSphere where the requirement was to create ISCSI data store for HA & DRS (High Availability & Distributed Resource Scheduler).  We created storage from windows server instead of having a dedicated storage appliance/VM. The purpose of such configuration is only for learning & testing the HA/DRS activity.

This configuration is designed on windows 2012 R2 with ESX 6.0.

On windows server, install ‘File and ISCSI services’ and make sure ‘ISCSI Target server’ is selected.

1.jpg

Once the role is installed then select ‘File and storage services’ feature and then ‘ISCSI’.

Right click on empty space and select ‘New ISCSI Virtual Disk’ as shown in above example. In my screenshot one of the volume is created and I am creating a new one.

2.jpg

Now select the partition where you wish to place the Virtual Disk.

3.jpg

Give the appropriate Name and Description and confirm the path of SCSI target.

4.jpg

Give the appropriate size & select type of disk as mentioned below. In my example I have selected ‘Dynamically expending’ as I don’t need to worry about space.

5.jpg

You need to create target that will be the list of ESX servers participating in HA/DRS and access VHD using ISCSI protocol.

6.jpg

In my example, I have used DNS name of ESX servers as initiator but it can also be access using IQN, IP address OR MAC address.

16.jpg

In following screenshot, there are two ESX servers are added but you have even more.

8.jpg

Select type of authentication, I am leaving it blank to avoid confusion.

9.jpg

In next page, you can confirm the setting you have selected and the result of different section will be available for you.10.jpg

Now the shared ISCSI disk is ready, you can add this in ESX server using VSphere console. Select ESX server \ Manage \Storage \ Storage Adaptor \Target \ Add Target.

11.jpg

Rescan storage so all newly attache drives are visible.

12.jpg

Now you should be able to see the path of all available SCSI share disk.

13.jpg

SCSI disk will also be available as storage devices.

14.jpg

In your windows server, you would notice the target status as ‘connected’.

15.jpg

#disaster-recovery, #esx, #scsi, #storage, #vmware

Overview of SQL 2016 ‘Always Encrypted’ Security feature.

In couple of my last blogs, we discussed about various security options with SQL 2016. Now in this blog, we are going to discuss & demonstrate one other important security feature of SQL 2016 that is called as ‘Always Encrypted’.

The legacy features provide encryption either on network layer (SSL) OR data files (MDF) OR Backup files(BAK) and encryption happens for all data OR none. But ‘Always encrypted’ uses client application layer via ADO.Net to encrypt/decrypt column specific data. You may have table where ‘FirstName’ & ‘LastName’ is in plain text but the ‘passport no’ is encrypted. The ‘passport no’ will always be in cipher text during transition via network and will save in encrypted format in data files OR backup files. This feature also available for SQL azure so even Microsoft will see ecrypted data.

Let’s use following steps to demonstrate how Always encrypted can be configured & use in test/production environments of SQL 2016.

---Create AlwaysEncrptedDemo database. 
USE [master]
CREATE DATABASE AlwaysEncryptedDemo
GO
---Create table with 6 columns.   
USE [AlwaysEncryptedDemo]
CREATE TABLE EmployeeData
(
       EMPID INT Primary key,
       FirstName VARCHAR(100), 
       LastName VARCHAR(100),
       MobileNo BIGINT, 
       SALARY BIGINT,
       PassportNo VARCHAR(100)
)
GO

-- Insert 5 records. 
USE [AlwaysEncryptedDemo]
INSERT INTO EmployeeData VALUES
       (1, 'Pradeep', 'Papnai', 9595046647, 50000, 'ATNPP12345'), 
       (2, 'Vikram', 'Jadhav', 8088145698, 52000, 'YAG8383P'),
       (3, 'Darshan', 'Bandodkar', 9198234567, 51000, 'DD123453'),
       (4, 'Ajay', 'Jain', 875691232, 55000, 'AJ123JK12'),
       (5, 'Yogesh', 'Ghute', 8787453212, 49000, 'PT9876KA12')

Normal retrieval of rows.

-- Retrieve all records, our next action will encrypt last three columns those are confidential. 
USE [AlwaysEncryptedDemo]
SELECT EMPID, FirstName, LastName,MobileNo, SALARY, PassportNo
FROM EmployeeData

FirstSelect

Configure Always Encrypted on columns of a table

Please note these step can also be done using SQL query & Powershell but it’s recommended to do it using wizard for better understanding.

  1. Expand ‘AlwaysEncrptedDemo’ database, Right click on ‘EmployeeData’, select ‘Encrypt columns’
  2. Click NEXT on introduction page.
  3. Check on columns ‘MobileNo, Salary, PassportNo. Select EncrptionType click NEXT.

There are two type of encryption is supported:-

Deterministic – Always encrypts to the same cipher text means decrypted data will always be the same and data can be indexed.

Randomized – It’s consider as more secure as the cipher text will always be different.  The data of this column cannot be indexed.

  1. Key Store provider = Windows Certificate Store, Master key source = current user, click NEXT.
  2. Select ‘Proceed to finish now’, although you can generate PowerShell next time you do it, click NEXT.
  3. Verify the choice you made, click ‘NEXT’.
  4. Encryption might take time due to size of column present in table. In production system, it is usually performed off peak hours.

Here is the slide show for above mentioned steps:-

This slideshow requires JavaScript.

-- Now if you run select query again the results for column MobileNo, Salary, PassportNo will be encrypt
USE [AlwaysEncryptedDemo]
SELECT EMPID, FirstName, LastName, MobileNo, SALARY, PassportNo
FROM EmployeeData

EncryptSelect

If you wish to see encrypted data using SQL Management studio then the connection string should include following line ‘column encryption setting=enabled’

Right click on query windows, connection-change connection, Additional Connection Parameters

SSMS_Connection_string

-- Now the columns can be seen in without encryption. 
USE [AlwaysEncryptedDemo]
SELECT EMPID, FirstName, LastName, MobileNo, SALARY, PassportNo
FROM EmployeeData

FirstSelect

Well this is not the end of story. If you connect to this SQL server using SSMW from other machine you will notice columns are encrypted even though connection string encrypted is enabled, it’s because you don’t have certificate for encryption.

Always encrypted feature maintain two keys :-

Column Master Key – this is an encryption key that protects the column encryption key(s). You must have at least one master key before encrypting any columns.

Column Encryption Key – this is the encryption key that actually protects that encrypted columns.

Keys

You can either use existing keys OR create a new one. In order to copy the certificate, use following steps.

Click on ‘RUN’, Type ‘MMC’ and press the ENTER key. On the ‘File’ menu, click on ‘Add/Remove Snap In’.
Select ‘Certificates’. Select ‘My user account’ and click ‘Finish’ and ‘OK’.
Expand ‘Personal’, then ‘Certificate’, Select ‘Always Encrypted auto certificate’

Certificate 1

You can export this certificate to other machine where ever client OR application connects to SQL servers for encryption purpose. Here is example how connection string can be used with PowerShell.

Use case without encryption connection string

#Change SQL server name
$connectionString = "Data Source=SQL1;Integrated Security=true;Initial Catalog=AlwaysEncryptedDemo;"
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$query = “SELECT EMPID, FirstName, LastName,MobileNo, SALARY, PassportNo FROM EmployeeData”
$command = $connection.CreateCommand()
$command.CommandText = $query
$result = $command.ExecuteReader()
$table = new-object "System.Data.DataTable"
$table.load($result)
$table | format-table 
$connection.Close()

Powershell_Without_S.jpg

Use case with encryption connection string

$connectionString = "Data Source=SQL1;Integrated Security=true;Initial Catalog=AlwaysEncryptedDemo;column encryption setting=enabled"
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$query = “SELECT EMPID, FirstName, LastName,MobileNo, SALARY, PassportNo FROM EmployeeData”
$command = $connection.CreateCommand()
$command.CommandText = $query
$result = $command.ExecuteReader()
$table = new-object "System.Data.DataTable"
$table.load($result)
$table | format-table 
$connection.Close()

With_connection_string.jpg

More Details

#sql, #sql-security, #sql2016