Introduction (Analytics Platform System)

[image]

Introduction (Analytics Platform System)
Send Feedback

Welcome to the product documentation for Microsoft® Analytics Platform System Appliance Update 3.

Help file version: Appliance Update 3, Date: 2014-12-09

Microsoft Analytics Platform System

Microsoft Analytics Platform System (APS), a data platform designed for data warehousing and Big Data analytics, offers deep data integration, high-speed query processing, highly scalable storage, and simple maintenance for your end-to-end business intelligence solutions.

Analytics Platform System architecture

Analytics Platform System hosts both Microsoft SQL Server Parallel Data Warehouse (PDW) and Microsoft HDInsight (HDI) workloads, all in the same appliance.

PolyBase technology combines relational PDW data with Hadoop data from multiple sources including Hortonworks on Windows Server, Hortonworks on Linux, Cloudera on Linux, HDInsight’s Windows Azure blob storage, and HDInsight on Analytics Platform System. These advanced data integration abilities, plus deep integration with Business Intelligence tools, allow Analytics Platform System to return integrated analysis that enables your business decision makers to make better and more insightful business decisions.

Analytics Platform System ships to your data center as an appliance with hardware and software pre-installed and pre-configured to run multiple workloads. When you purchase Analytics Platform System, you purchase Compute nodes for PDW and optionally purchase additional nodes for HDInsight according to your business requirements.

Analytics Platform System is not only fast and scalable, it is designed with high redundancy and high availability, making it a reliable platform you can trust with your most business critical data. Analytics Platform System is designed for simplicity which makes it easy to learn and to manage. PDW’s PolyBase technology for analyzing Hadoop data, and its deep integration with Business Intelligence tools make it a comprehensive platform for building end-to- end solutions.

HDInsight

HDInsight is a 100 percent Apache Hadoop-based data platform that provides a Hadoop-based cloud solution for Windows Azure services, and an on-premises solution for Analytics Platform System. HDInsight manages data of any type, whether relational or nonrelational, and of any size. With APS, you can store and analyze HDInsight data on your own premises.

With APS, you can perform more complex analysis faster by using SQL Server PDW’s PolyBase to integrate analysis of your HDInsight non-relational data and your PDW relational data, all on the same appliance.

HDInsight, running on APS, makes it easy to:

  • Load data of any structure into PDW. By first loading your data into HDInsight, you can transform and process it, and then use PDW’s PolyBase to quickly load the data into PDW relational tables.

  • Perform fast integrated analysis of HDInsight data and PDW data. The data are stored within close proximity of each other on the same appliance, making it fast to run integrated analysis with PDW’s PolyBase.

  • Export relational data to HDInsight region for a cold data store. To reduce the size of data stored in SQL Server PDW, you can move it to HDInsight and query the data at any time.

  • Trust that your data will be protected with the Analytics Platform System built-in high availability.

  • Monitor and manage HDInsight by using centralized appliance management tools.

For information, see HDInsight (Analytics Platform System) in this documentation and HDInsight benefits on the Windows Azure web site.

SQL Server Parallel Data Warehouse

Use SQL Server Parallel Data Warehouse as the core relational data warehousing component of your end-to-end business intelligence solutions. With PDW’s massively parallel processing (MPP) design, queries commonly complete 50 times faster than traditional data warehouses built on symmetric multi-processing (SMP) database management systems.

Note Note

50 times faster means that queries complete in minutes instead of hours, or seconds instead of minutes. With this breakthrough performance, your business analysts can generate more extensive results faster, and can easily perform ad-hoc queries or drill down into the details. As a result, your business can make better decisions, faster.

In addition to achieving breakthrough query performance, PDW makes it easy to:

  • Grow your data warehouse to anywhere from a few terabytes to over 6 petabytes of data in a single appliance by adding “scale units” to your existing system

  • Trust your data will be there when you need it because of the built- in high redundancy and high availability,

  • Solve modern data challenges of loading and consolidating data

  • Integrate Hadoop data with relational data for fast analysis by using PDW’s highly parallelized PolyBase technology

  • Use Business Intelligence tools to build comprehensive end-to-end solutions.

For more information about the benefits of PDW, see the whitepaper A Breakthrough Platform for Next-Generation Data Warehousing and Big Data Solutions on MSDN.

Send feedback on this topic to Microsoft.

© 2014 Microsoft Corporation. All rights reserved.


How to Use This Documentation (Analytics Platform System)

[image]

How to Use This Documentation (Analytics Platform System)
Send Feedback

Information to help you use the Analytics Platform System documentation successfully.

Send feedback

We welcome your feedback on this documentation. Please send your comments to the authors, Barbara Kess and Rick Byham, at pdwdoc@microsoft.com.

How to use this documentation

To use this documentation file most effectively:

  1. Use the Table of Contents to see the logical structure of the topics.

    Table of Contents for SQLServerPDW.chm

  2. Use the Index to find topics by name or category.

    Index for SQLServerPDW.chm

  3. Use Search to find all topics that contain a specific word of phrase. Use double-quotes around phrases.

    Search in SQLServerPDW.chm

We welcome your feedback about this documentation! For best results when sending feedback, please send feedback directly to pdwdoc@microsoft.com instead of using the Feedback link at the top of individual topic pages.


About This Release (Analytics Platform System)

[image]

About This Release (Analytics Platform System)
Send Feedback

Find out information specific to this release of Analytics Platform System.

In this section

  • What's New (Analytics Platform System)

  • Known Issues (Analytics Platform System)

  • Breaking Changes (Analytics Platform System)

  • Versions (Analytics Platform System)

  • Legal Notice for Documentation (Analytics Platform System)


What's New (Analytics Platform System)

[image]

What's New (Analytics Platform System)
See Also Send Feedback

Learn about what’s new in Microsoft® Analytics Platform System, a data platform that hosts SQL Server Parallel Data Warehouse (PDW) and HDInsight (HDI) workloads all on the same appliance.

Contents

Appliance Update 3

  • Documentation and Tools are Public

  • Transact-SQL Error Handling

  • Software Versions

  • PolyBase Supports ORC Files

  • Microsoft Data Management Gateway

  • Appliance Domain Simplification

Appliance Update 2

  • HDInsight Workload

  • Transact-SQL Compatibility

  • System Views

  • PolyBase Enhancements

  • Send Feedback to Microsoft

  • SQL Server database tooling (previously called SSDT)

  • Integration Services (SSIS) 2014 destination adapters

Appliance Update 1

  • New HDInsight Workload

  • PolyBase for External Data Integration

  • Using SQL Server PDW Apliance Update 1

Before you upgrade to the latest release, see:

  • Breaking Changes (Analytics Platform System)

  • Known Issues (Analytics Platform System)

AU3: Documentation and Tools are Public

This documentation file and the client tools are now available publicly on the Microsoft Download Center.

AU3: Transact-SQL Error Handling

Functions and statements for error handling are now supported. See @@ERROR, ERROR_MESSAGE, ERROR_NUMBER, ERROR_PROCEDURE, ERROR_SEVERITY, ERROR_STATE, PRINT, RAISERROR, THROW, TRY...CATCH, XACT_STATE.

AU3: Software Versions

APS now runs on Windows Server 2012 R2. User and system data is stored in SQL Server 2014 databases.

AU3: PolyBase Supports ORC Files

PolyBase now supports the Optimized Row Columnar (ORC) file format for external files stored on HDFS. This option requires Hive version 0.11 or higher. In conjunction with Hortonworks Data Platform (HDP) 2.0, ORC file formats offer better compression and performance than the RCFILE formats. To use the ORC file, see CREATE EXTERNAL FILE FORMAT (SQL Server PDW)

AU3: Microsoft Data Management Gateway

Now you can use Microsoft Data Management Gateway to enable Power BI scenarios on PDW. For example, Power Query and Power Map for Excel, and Power BI Q&A can now access on-premises PDW data through secure OData feeds. Data Management Gateway is pre-installed on the Control node and pre-configured by Microsoft Support. To configure Power BI to access PDW, see Connect with Power BI or Power Query by using Data Management Gateway (Analytics Platform System).

AU3: Appliance Domain Simplification

The APS appliance now has one domain. For high availability, there are two replicated Active Directory domain controllers that run on the orchestration host and the passive host. There is no longer a need for the domain controller to failover to a passive host; if one fails, the other is already online.

This change has enabled flexibility for all virtual machines configured for failover. Since the domain controller does not failover, there is no longer a need for all of the virtual machines running on the same host to failover. As a result, the corralling service is no longer needed to set the start order for virtual machines. Now, a Control node, Compute node, or appliance fabric virtual machine can fail to any available passive host.

DNS entries and connection strings pointing to the PDW domain must be redirected to the appliance domain. For example a pointer to PDW_domain-CTL01.PDW_domain.pdw.local should be adjusted to be PDW_region-CTL01.appliance_domain.local.

For more information and some diagrams, see PDW and Appliance Fabric Physical Components (Analytics Platform System).

AU2: HDInsight Workload

The HDInsight region is updated to Hortonworks Data Platform (HDP) 2.0. The YARN project is introduced as part of Hadoop 2.0. It takes over responsibility for all cluster resource management from the MapReduce project. For further details about YARN, see the Hortonworks web site.

For updated port specifications for external data sources with HDP 2.0, see CREATE EXTERNAL DATA SOURCE (SQL Server PDW).

AU2: Transact-SQL Compatibility

User-Defined Schemas

User-defined schemas are supported. For more information, see CREATE SCHEMA (SQL Server PDW).

Statistics

The following statistics elements are now supported:

  • Filtered statistics are in CREATE STATISTICS (SQL Server PDW).

  • Sampling by PERCENT is in CREATE STATISTICS (SQL Server PDW). You can use this option on both regular and external tables.

  • DBCC SHOW_STATISTICS (PDW) shows the header, density vector, and histogram for Control node statistics.

  • STATS_DATE (PDW) is added to show the date of the most recent Control node statistics.

Functions

The following functions are now supported: ACOS, ASCII, ATN2, CHAR, CONCAT, COT, DATALENGTH, DATEFROMPARTS, DATENAME, DATETIMEFROMPARTS, DATETIME2FROMPARTS, DATETIMEOFFSETFROMPARTS, DEGREES, DIFFERENCE, EOMONTH, GETUTCDATE, HASHBYTES, ISDATE, NCHAR, PARSENAME (SQL Server PDW), PI, RADIANS, REVERSE, SMALLDATETIMEFROMPARTS, SOUNDEX, SPACE, SQL_VARIANT_PROPERTY, STUFF, SWITCHOFFSET, SYSDATETIME (SQL Server PDW), SYSDATETIMEOFFSET, SYSUTCDATETIME, TERTIARY_WEIGHTS, TIMEFROMPARTS, TODATETIMEOFFSET, TYPE_ID, TYPE_NAME, TYPEPROPERTY, UNICODE, and ODBC Scalar Functions.

The DAX functions EXACT and REPLACE are now supported in DirectQuery operations against a PDW data source.

AU2: System Views

sys.pdw_nodes_pdw_physical_databases (SQL Server PDW) was added in AU1, but was not mentioned in the docs until now.

AU2: PolyBase Enhancements

PolyBase now supports connections to Hortonworks (HDP 2.0) for Windows Server, HDInsight on Analytics Platform System (version AU2) (HDP 2.0), Azure blob storage on Microsoft Azure (WASB[S]), Hortonworks (HDP 2.0) for Linux. For more information, see sp_configure (SQL Server PDW).

A new query hint (EXTERNALPUSHDOWN) can force or disable the pushdown of the computation of qualifying expressions or operators in Hadoop. For more information, see OPTION (SQL Server PDW).

AU2: Send Feedback to Microsoft

Analytics Platform System has an optional telemetry feature that sends Admin Console data to Microsoft. We encourage you to enable this to help us improve the product. For more information, see Send Telemetry Feedback to Microsoft (SQL Server PDW).

AU2: Get the latest SQL Server database tooling (formerly called SSDT)

SQL Server database tooling, formerly called SQL Server Data Tools, is available in Visual Studio 2012 and 2013. AU2 support is included in the July 2014 update for both versions.

To install the updates, open Visual Studio and go to the Tools > Extensions and Updates… menu. In the Extensions and Updates dialog, click on Updates. Depending on which updates you have already applied, you could see Microsoft SQL Server Update for database tooling and Visual Studio 2013 Update 3 in the list.

If you have Visual Studio 2013 Ultimate, Premium, or Professional, you need to update SQL Server database tooling component. If you have Visual Studio 2013 Express for Windows Desktop, you need to apply Visual Studio Update 3 in addition to the SQL Server database tooling update. We recommend that you install Visual Studio Update 3, regardless of the edition you have to ensure you have the latest bug fixes from Visual Studio.

Install the updates you need. Restart Visual Studio after updates are applied if it was not closed during the update.

If you have Visual Studio 2012, install the latest download of SQL Server data tooling from SQL Server Data Tools for Visual Studio 2012.

AU2: Integration Services 2014 destination adapters

Integration Services 2014 destination adapters are available in the C:\PDWINST\ClientTools directory on the Control node. For more information, see Install Integration Services Destination Adapters (SQL Server PDW).

[ TOP ]

AU1: New HDInsight Workload

SQL Server PDW has evolved into Microsoft Analytics Platform System, a single fabric that can host both SQL Server PDW and Microsoft’s HDInsight running on separate regions of nodes, all within the same appliance.

Microsoft’s HDInsight is a 100 percent Apache Hadoop-based data platform that manages data of any type, whether relational or nonrelational, and of any size. By having the HDInsight region, you can store and analyze HDInsight data on your own premises, integrate SQL Server PDW’s relational data with HDInsight’s non-relational data, and integrate your on-premises HDInsight data with Windows Azure blob storage. Business intelligence solutions can quickly integrate relational and non-relational data that is stored on the same appliance since the appliance InfiniBand supports fast data transfer up to 56 Gb/s, and the data transfers in parallel between HDInsight’s DataNodes and PDW’s Compute nodes.

The HDInsight workload is optional. Ask your sales consultant about pricing and licensing options for each workload.

For more information about HDInsight, see HDInsight (Analytics Platform System) and HDInsight Topology (Analytics Platform System).

[ TOP ]

AU1: PolyBase for External Data Integration

Queries perform predicate pushdown

For performance improvements when querying external tables, PolyBase now performs predicate pushdown for queries on external Hadoop tables. To do this, SQL Server PDW issues map/reduce jobs to pre-compute some query predicate expressions in order to reduce the size of the data that must be copied to SQL Server PDW for query execution. For more information, see Queries on External Tables With Predicate Pushdown (SQL Server PDW)

New configuration options

You can now import and export data between SQL Server PDW and Windows Azure blob storage (WASB). In addition to the external Hadoop platforms supported in the previous release, you can use HDInsight on Analytics Platform System as a Hadoop source.

PolyBase now supports these external data sources:

  • Hortonworks on Windows Server (HDP 1.3 and HDP 2.0)

  • Hortonworks on Linux (HDP 1.3 and 2.0)

  • Cloudera CDH 4.3 on Linux

  • HDInsight’s Windows Azure blob storage (WASB[S])

  • HDInsight on Analytics Platform System

To configure Hadoop and Windows Azure blob storage connectivity for the appliance, see sp_configure (SQL Server PDW).

Statistics on external tables

You can now create and drop statistics on external tables. To use statistics with external tables, simply use the name of an external table when you call CREATE STATISTICS (SQL Server PDW), and DROP STATISTICS (SQL Server PDW). For more information, see Understanding Query Optimization Statistics (SQL Server PDW).

New statements and syntax changes

To provide a framework for external data, this release includes two new data definition language (DDL) statements, CREATE EXTERNAL DATA SOURCE (SQL Server PDW) and CREATE EXTERNAL FILE FORMAT (SQL Server PDW), that create a data source object and a file format object respectively.

The syntax for CREATE EXTERNAL TABLE (SQL Server PDW) and CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW) has changed to refer to these new objects. The previous syntax for these two statements is deprecated in this release. You can continue to use the previous syntax, but need to update scripts to use the new syntax in preparation for the next release.

New DDL statements:

  • CREATE EXTERNAL DATA SOURCE (SQL Server PDW)

  • CREATE EXTERNAL FILE FORMAT (SQL Server PDW)

  • DROP EXTERNAL DATA SOURCE (SQL Server PDW)

  • DROP EXTERNAL FILE FORMAT (SQL Server PDW)

Modifed DDL statements

  • CREATE EXTERNAL TABLE (SQL Server PDW)

  • CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW)

New system views

  • sys.external_tables (SQL Server PDW) (formerly sys.pdw_external_tables)

  • sys.external_file_formats (SQL Server PDW)

  • sys.external_data_sources (SQL Server PDW)

  • sys.dm_pdw_hadoop_operations (SQL Server PDW)

  • sys.dm_pdw_dms_external_work (SQL Server PDW)

[ TOP ]

AU1: Using SQL Server PDW Appliance Update 1

SQL Server Parallel Data Warehouse Appliance Update 1 now runs as one of the workloads on Analytics Platform System. PDW runs on an enhanced version of SQL Server 2012 Service Pack 1 that includes updates for in-memory-optimized clustered columnstore indexes. It also includes the new cardinality estimator that is in SQL Server 2014.

Adding PDW scale units is faster

Adding SQL Server PDW scale units to the appliance is faster and requires less downtime than the previous release. The downtime is reduced because SQL Server PDW can stay online while a Microsoft support engineer configures the new scale units, and because it is no longer necessary to backup and restore SQL Server PDW databases in order to redistribute the data after adding scale units. When the new scale units are ready for data, SQL Server PDW automatically redistributes the existing data to the additional nodes.

There is still downtime for adding scale units since the appliance must be completely shut down to physically add the new hardware, and PDW must be offline while it redistributes user data to all the Compute nodes. The time required for the redistribution depends on the size of the data.

The redistribution requires some free space to exist in the existing SQL Server PDW region before you add scale units. The free space required is approximately the size of the largest existing table. Contact Microsoft support to find out how much free space you have and how much is required to expand your capacity.

[ TOP ]

Integrated Authentication

SQL Server PDW now supports Windows authentication logins. For more information, see Security - Windows Authentication and SQL Server Authentication (SQL Server PDW).

Transparent Data Encryption

SQL Server PDW now supports transparent data encryption. For more information, see Transparent Data Encryption (SQL Server PDW).

To support TDE, a master key and certificates can now be created in the master database. A database encryption key can be created in a user database. For more information, see CREATE MASTER KEY (SQL Server PDW), CREATE CERTIFICATE (SQL Server PDW), and CREATE DATABASE ENCRYPTION KEY (SQL Server PDW). Additional DMV's and stored procedures are added to support TDE. For more information, see Transparent Data Encryption (SQL Server PDW).

Appliance Domain Model

The appliance domain configuration has changed to support HDInsight and Windows authentication. For more information, see PDW Domain Security (SQL Server PDW), Security - Configure Domain Trusts (SQL Server PDW), and Understanding the Security Model of the HDInsight Region (Analytics Platform System).

dwloader Loads Data Directly to the Compute Nodes

dwloader now loads data directly to each of the Compute nodes without passing the data through the Control node. To load data, dwloader communicates with the Control node to gain network access to the Compute nodes, and then it transfers data in a round-robin manner to the Compute nodes.

Data Movement Service (DMS) on each Compute node receives the raw data from dwloader, processes it, and then moves the processed data in parallel to the correct Compute node instance of SQL Server.

For more information, see Load (SQL Server PDW).

dwloader uses less restrictive data type conversion rules

dwloader Command-Line Loader (SQL Server PDW) now aligns more closely with SQL Server behavior for reading input data from the input file and converting it to the target data type. The new behavior enforces the specified order of year, month, and day, but allows flexibility for how each is formatted.

To specify a date format use one of the new parameters for the –D option.

-D { mdy | myd | ymd | ydm | dmy | dym | custom_date_format }

The new conversion rules are highly compatible with the previous release, however, there are some differences. Please be sure to read the Breaking Changes (Analytics Platform System) to see a list of differences in behavior.

[ TOP ]

dwloader batch size is calculated dynamically

You do not need to specify the batchsize (-b batchsize) in dwloader. Beginning with SQL Server 2012 PDW, the Control node dynamically computes the batch size for the bulk copy that DMS performs into SQL Server instances on the Compute nodes.

The –B batchsize parameter should only be used at the suggestion of Microsoft Support. For more information, see dwloader Command-Line Loader (SQL Server PDW).

SQL Server Data Tools update

To use SQL Server Data Tools with SQL Server 2012 PDW Appliance Update 1, install the January 2014 release, or later, of SQL Server Data Tools for Visual Studio 2012 from the Microsoft SQL Server Data Tools download page on MSDN.

In the January release, SQL Server Data Tools for Visual Studio 2012 is the only supported version for SQL Server PDW Appliance Update 1. SQL Server Data Tools for Visual Studio 2010 is no longer supported. Note that Visual Studio 2013 does not require a separate download because it already has SQL Server Data Tools integrated; however, SQL Server Data Tools in Visual Studio 2013 does not yet have support for SQL Server PDW. This will change in an upcoming release of Visual Studio 2013.

The January release of SQL Server Data Tools for Visual Studio 2012 supports SQL Server 2012 PDW, SQL Server PDW Appliance Update 0.5, and SQL Server PDW Appliance Update 1. When connected to Appliance Update 1, you will see these new features:

  • For PolyBase, there is now an External Resources folder in SQL Server Object Explorer that shows the new external data source and external file format objects.

  • Logins for Windows users and groups are now visible in the object explorer under [Database Instance]->Security->Logins.

  • The security objects Master Key, Certificates, and Database Encryption Keys are now visible in object explorer.

For more information, see Install SQL Server database tooling for Visual Studio (SQL Server PDW).

Upgrade sets all databases to compatibility level 110

After upgrading to SQL Server 2012 PDW Appliance Update 1, all databases will use database compatibility level 110, which is the compatibility level for SQL Server 2012. This ensures that all databases will be able to use all of the SQL Server 2012 functionality that PDW uses. For example, SQL Server 2008 R2 databases cannot use all of the SQL functions, such as PERCENTILE_CONT, and the new cardinality estimator until they are upgraded to compatibility level 110.

Note Note

Note, both SQL Server 2012 PDW and SQL Server 2012 PDW Appliance Update 1 have the benefits of using the new cardinality estimator that is in SQL Server 2014.

Databases that were created on SQL Server 2012 PDW already have compatibility level 110. With this change, any existing databases that were created with SQL Server 2008 R2 PDW will be set to compatibility level 110 after the upgrade. Also, any SQL Server 2008 R2 PDW databases that are restored to SQL Server 2012 PDW Appliance Update 1 will have compatibility level 110 after the restore.

See Also

Concepts

Breaking Changes (Analytics Platform System)
Known Issues (Analytics Platform System)


Known Issues (Analytics Platform System)

[image]

Known Issues (Analytics Platform System)
See Also Send Feedback

Describes known issues in Analytics Platform System. These are temporary issues that will be fixed or will no longer be applicable in a future release.

Contents

  • Appliance Update 3

  • Appliance Update 2

  • Appliance Update 1

Appliance Update 3

Database compatibility

For appliances that are upgraded from AU2 to AU3:

  • A database that is upgraded from AU2 to AU3 will continue to have database compatibility level 110.

  • A database that is created new in AU3 will have database compatibility level 110.

  • Any database that is restored to AU3 will have database compatibility level 120.

For an appliance that has a clean install of AU3, all databases that are created or restored will have database compatibility level 120.

Appliance Update 2

These are the known issues for Appliance Update 2.

DENY ON DATABASE is blocked

PDW sometimes executes stored procedures to distribute user actions to the compute nodes. Therefore, the execute permission for an entire database cannot be denied. (For example DENY EXECUTE ON DATABASE::<name> TO <user>; will fail.) As a work around, deny the execute permission to user-schemas or specific objects (procedures).

Appliance Update 1

These are the known issues for Appliance Update 1.

Verify actual data row size before loading data with Integration Services

dwloader and Integration Services use different code paths when loading data into SQL Server PDW. It is possible to use Integration Services to load a row into SQL Server PDW that exceeds the DMS buffer size and will cause a failure when DMS tries to move the row.

To avoid this error, use dwloader to load data, or verify the length of your actual data before loading it with Integration Services. For more information, see the Table, Bytes per row, DMS buffer defined size description and the examples in Minimum and Maximum Values (SQL Server PDW).

Statement execution delayed by locks

SQL Server prevents new lock requests from blocking older statements which are waiting for locks. SQL Server PDW has not fully implemented this process. In SQL Server PDW, continuous requests for new shared locks can sometimes block an older request for an exclusive lock. For example, an UPDATE statement requiring an exclusive lock, can be blocked by a series of shared locks granted for SELECT statements.

To resolve a blocked process (identified by reviewing the sys.dm_pdw_waits (SQL Server PDW) DVM), stop submitting new requests until the exclusive lock has been satisfied.

Restrictions on ROLAP operations

In this release of SQL Server PDW, some ROLAP operations are not supported. For more information, see Connect With SQL Server Analysis Services (SQL Server PDW).

See Also

Concepts

What's New (Analytics Platform System)


Breaking Changes (Analytics Platform System)

[image]

Breaking Changes (Analytics Platform System)
Send Feedback

These are breaking changes that apply when upgrading to the Analytics Platform System Appliance Updates.

Contents

  • Appliance Update 3

  • Appliance Update 2

  • Appliance Update 1

Appliance Update 3

Existing Queries on External Tables Might Return Different Results

In AU2, by default a PolyBase query on an external table folder returned data stored in the root folder only. In AU3, by default a query on an external table now returns data from the root folder and all of the subfolders. The root folder is specified by the LOCATION parameter in CREATE EXTERNAL TABLE.

To avoid unwanted results, verify that subfolders in external file folders do not contain extra data that should not be returned in query results.

For backwards compatibility, your appliance administrator can change the default setting to read data from only the root folder of the external file location. To do this, set <polybase.recursive.traversal> to 'false' in the appliance configuration file called core-site.xml.

Appliance Update 2

These breaking changes apply when upgrading to Appliance Update 2.

Default Schemas May Change Behavior

PDW now supports user-schemas and default schemas for users. Previous scripts that assumed all objects were created in the dbo schema, might have unexpected results if the users default schema is not dbo.

Appliance Update 1

These breaking changes apply when upgrading to Appliance Update 1.

Changes to SQL Server Data Tools

Update SQL Server Data Tools to the January 2014 (or later) release of SQL Server Data Tools for Visual Studio 2012. SQL Server Data Tools is no longer supported for Visual Studio 2010.

For more information, see Install SQL Server database tooling for Visual Studio (SQL Server PDW)

Changes to input date formats for loading data with dwloader

Loading date and time data can have difference results in Appliance Udpate 1 than the previous release. dwloader enforces only the order of month, day, and year when loading datetime data from the input file; it allows, but no longer enforces custom date formats.

Data that will load in Appliance Update 1, but would be rejected in previous release:

  • For the custom date format 'yyyy-mm-dd', dwloader only enforces the order ymd. dwloader will load the incoming data '99-2-1' whereas this would be rejected in the previous release.

  • For empty strings or strings that contain only spaces, dwloader will load the value 1900-01-01 12:00:00AM whereas this would be rejected in the previous release.

  • For dates that do not contain separators, dwloader will load them whereas they would be rejected in the previous release. For example, '19991231' will be loaded as '1999-12-31'.

  • For missing date values, dwloader will insert 1900-01-01 whereas this would be rejected in the previous release.

  • For missing time values, dwloader will insert 00:00:00 whereas this would be rejected in the previous release.

  • For time data that does not specify AM or PM, dwloader will load this by using a 24-hour clock. For example, 1999/01/06 20:00:00 will be loaded as 20:00:00 (8:00 PM), whereas in the previous release, this would be loaded as 8:00:00 (8:00 AM).

  • For input data formatted as HH:mm:ssZ, dwloader will load the value into time, datetime2, or datetimeoffset data, whereas this would be rejected in the previous release.

  • For input data that contains only the year, dwloader will load the value January 1. For example, 1999 will load as 1999-01-01. Previously, this would be rejected.

  • When loading into a smalldatetime column, dwloader will round up seconds and fractional seconds, whereas this would be rejected in the previous release. For example, 1999-01-05 20:10:35.123 will load as 01-05 20:11. Previously, this would be rejected.

  • When loading into a time column, dwloader will ignore date dataFor example, 1999-01-01 12:12 will be inserted into the time column as 12:12. Previously, this would be rejected.

  • For input data with leading or trailing spaces, dwloader will ignore the spaces and load the data. Previously, this would be rejected.

  • Input file rows can have the EOL characters '\r\n', '\r', or '\n'. Previously, input file rows that ended in only '\r', or '\n' would be rejected.

Data that will not load in Appliance Update 1, but passed in previous release:

  • Data ordered as ydm is supported only for loading into columns of datetime and smalldatetime. Ydm data cannot be loaded into columns of datetime2, date, and datetimeoffset, whereas previously this would pass.

Update scripts that use CREATE EXTERNAL TABLE before upgrading to the next appliance update.

The CREATE EXTERNAL TABLE (SQL Server PDW)and CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW) syntax has changed to use data source and file format objects. The previous syntax is deprecated in Appliance Update 1 and will be removed in the next appliance update.

Update scripts that use the former syntax to use the new syntax so that future upgrades will not break. When upgrading to Appliance Update 1, SQL Server PDW will re-create all existing external tables to use the new syntax and objects.

Similarly, you can continue to use the former syntax in Appliance Update 1. In this case, SQL Server PDW will convert the DDL to the new syntax and will auto-create the data source and file format objects.

The auto-created objects follow the naming convention DataSource_GUID and FileFormat_GUID for data sources and file formats respectively.

Update scripts that use the sys.pdw_external tables system view

The system view sys.pdw_external_tables has been removed from Appliance Update 1. The information is now contained across these three system views:

  • sys.external_tables

  • sys.external_file_formats

  • sys.external_data_sources

Java will be removed and upgraded automatically

You no longer need to manually install Java onto the appliance. Upgrading SQL Server 2012 PDW to Appliance Update 1 will install Oracle Java SE Development Kit 7 on the SQL Server PDW nodes for use with PolyBase. Note that the upgrade will remove any existing non-Oracle Java versions from the appliance.

SET ROWCOUNT does not affect some statements

SET ROWCOUNT does not affect these statements: INSERT, UPDATE, DELETE, CREATE TABLE AS SELECT, CREATE REMOTE TABLE AS SELECT, and CREATE EXTERNAL TABLE AS SELECT. Modify applications that use this. For a similar behavior, use the TOP (SQL Server PDW) statement.

This was deprecated in the previous release, and is now in effect in Appliance Update 1.


Versions (Analytics Platform System)

[image]

Versions (Analytics Platform System)
Send Feedback

Versions and build numbers SQL Server Parallel Data Warehouse.

Build Numbers

This is a list of the build numbers for the releases of SQL Server Parallel Data Warehouse.

Product | Appliance Update | Build Number
APS SQL Server Parallel Data Warehouse | AU3 | 10.00.6186.00
APS SQL Server Parallel Data Warehouse | AU2 | 10.00.5108.01*
SQL Server 2012 Parallel Data Warehouse | AU1 | 10.00.4721.01*
SQL Server 2012 Parallel Data Warehouse | AU0.5 | 10.00.4422.00*
SQL Server 2012 Parallel Data Warehouse | RTM | 10.00.4176.00*
SQL Server 2008 R2 Parallel Data Warehouse | AU3.5 | 10.00.3595.00
SQL Server 2008 R2 Parallel Data Warehouse | AU3 | 10.00.3176.00

*This version runs SQL Server 2012 on the Control node and Compute nodes, and it uses some features that are in SQL Server 2014.

Some third party applications use the build number to define the interface, and so to maximize SQL Server compatibility, PDW supports SQL Server 2008 for client connections. Hence, the official build version is based on 10.00 which indicates a SQL Server 2008 build number.

Software Versions

Product | Appliance Update | Operating System | SQL Server User Databases | SQL Server Compatibility Level for Applications
APS SQL Server Parallel Data Warehouse | AU3 | Windows Server 2012 R2 | SQL Server 2014 | SQL Server 2008 R2
APS SQL Server Parallel Data Warehouse | AU2 | Windows Server 2012 | SQL Server 2012 + selected features from SQL Server 2014 | SQL Server 2008 R2
SQL Server 2012 Parallel Data Warehouse | AU1 | Windows Server 2012 | SQL Server 2012 + selected features from SQL Server 2014 | SQL Server 2008 R2
SQL Server 2012 Parallel Data Warehouse | AU0.5 | Windows Server 2012 | SQL Server 2012 + selected features from SQL Server 2014 | SQL Server 2008 R2
SQL Server 2012 Parallel Data Warehouse | RTM | Windows Server 2012 | SQL Server 2012 + selected features from SQL Server 2014 | SQL Server 2008 R2
SQL Server 2008 R2 Parallel Data Warehouse | AU3.5 | Windows Server 2008 | SQL Server 2008 R2 | SQL Server 2008 R2
SQL Server 2008 R2 Parallel Data Warehouse | AU3 | Windows Server 2008 | SQL Server 2008 R2 | SQL Server 2008 R2


Legal Notice for Documentation (Analytics Platform System)

[image]

Legal Notice for Documentation (Analytics Platform System)
Send Feedback

Microsoft Analytics Platform System

Microsoft Analytics Platform System Privacy Statement

© 2014 Microsoft Corporation. All rights reserved. This document is provided “as-is”. Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it. Some examples are for illustration only and are fictitious. No real association is intended or inferred.

This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.

Microsoft Logo | Java Logo
The PolyBase technology and HDInsight are powered by Java. | Java is a registered trademark of Oracle and/or its affiliates.


APS Architecture (Analytics Platform System)

[image]

APS Architecture (Analytics Platform System)
Send Feedback

Details about the Analytics Platform System architecture.

Microsoft Analytics Platform System (APS), a data platform designed for data warehousing and Big Data analytics, offers deep data integration, high-speed query processing, highly scalable storage, and simple maintenance for your end-to-end business intelligence solutions.

Analytics Platform System architecture

In This Section

  • Hardware Components (Analytics Platform System)

  • Hardware Configurations (Analytics Platform System)

  • PDW Processing and Storage Capacity (Analytics Platform System)

  • PDW High Availability (Analytics Platform System)


Hardware Components (Analytics Platform System)

[image]

Hardware Components (Analytics Platform System)
Send Feedback

Analytics Platform System hardware is architected with scalable components so that you buy the right amount of processing and storage according to your business requirements. When you order Analytics Platform System, you will need a combination of these core hardware components. Specific hardware vendors might use different naming conventions or have additional components.

Note Note

When you determine the capacity of the HDInsight Region, please take into consideration that you cannot expand the HDInsight region by simply adding more hardware. If you need to expand with this version, you can do this by adding hardware and then rebuilding the HDI region.

Contents

  • Rack & Network

  • Data Scale Unit

  • PDW Base Scale Unit

  • HDI Base Scale Unit

Rack & Network

APS components are all stored in one or more racks that fit into your data center. Each rack comes with power distribution units (PDUs), two InfiniBand switches, and two Ethernet switches.

APS Rack & Network

Data Scale Unit

A Data Scale Unit contains the data hosts and direct attached storage (DAS) for processing and storing user data. To add capacity you add Data Scale Units according to configurations that are supported by your hardware vendor. As the number of Data Scale Units grows, you need to add additional Rack & Network components, as necessary, to provide more power, network, and rack infrastructure.

data host

A data host is a server dedicated to processing user data. Each Data Scale Unit is dedicated to either the PDW Region or the HDI Region. PDW runs one Compute node on each data host. HDI runs two data nodes on each data host. For HP appliances, the Data Scale Unit has two data hosts. For Dell and Quanta appliances, the Data Scale Unit has three data hosts.

direct attached storage (DAS)

The direct attached storage is a pool of disks connected to the data hosts. All of the data hosts can access any of the disks. As part of the shared nothing architecture, the Compute nodes running on the data hosts do not share individual disks. However, for high availability, the storage access is shared and each of the data hosts can access any of the disks.

[ TOP ]

PDW Data Scale Unit Architecture - DELL and Quanta

Scalability Unit for the Appliance

PDW Data Scale Unit Architecture - HP

HP Storage Unit

Description

For PDW, a Data Scale Unit has one server (host) for each Compute Node and one direct-attached disk array that is attached with Serial Attached SCSI (SAS). Within the storage cabinet, the disk array is divided into two halves that each have redundant power supplies. Windows Server Storage Spaces manages user data by duplicating data across RAID 1 mirrored disk pairs. The disks in each disk pair are stored in different halves of the disk array.

The disk array also contains hot spare disks and a system disk. If a disk fails, Storage Spaces uses the good copy of the data on the functioning disk to rebuild a duplicate copy of the data on a hot spare. This is an important self-healing capability that helps to protect against data loss.

The total number of disks for the Compute nodes:

  • DELL has 96 disks = (3 servers) * (16 disks per server) * (2 for redundant disks).

  • HP has 64 disks = (2 servers) * (16 disks per server) * (2 for redundant disks).

  • In addition, each disk array has hot spare disks and a system disk.

For high availability, when a Compute node fails over it can still function and access its user data through the other host in the Data Scale Unit. At least one of the direct attached physical hosts MUST be functioning or data access to the storage is lost.

For disk sizes, the direct attached storage can have 1, 2, or 3 Terabyte disk drives. All Data Scale Units must have disks of the same size.

HDI Scale Unit Architecture

The HDI and PDW scale units use the same hardware, but have different hardware configurations. For example, HDInsight leverages HDFS replication for data redundancy. It stores multiple copies of each data block and therefore does not need to mirror each disk, whereas PDW uses a RAID hardware mirror for redundancy. Since the redundancy is handled by HDInsight, each server hosts two DataNodes that each write to 16 disks in the Storage Spaces disk arrays. When a disk fails, HDInsight uses an existing copy of the data to rebuild the failed data.

DELL Scale Unit

The DELL scale unit has 3 servers and 96 disks, plus a few system disks. Each server hosts 2 DataNodes that each write to 16 simple disks in the Storage Spaces disk arrays. Altogether there are 6 DataNodes in the scale unit.

HDInsight scale unit for datanodes

HP Scale Unit

The HPL scale unit has 2 servers and 64 disks, plus a few system disks. Each server hosts 2 Hadoop Data Nodes that each write to 16 simple disks in the Storage Spaces disk arrays. Altogether there are 4 Hadoop Data Nodes in the HP scale unit.

HP HDInsight Scale Unit

[ TOP ]

PDW Base Scale Unit

The PDW Base Scale unit contains the minimum number of brain-power hosts, data hosts, and direct attached storage that is required for a PDW Region. It includes these components.

orchestration host

This server runs the brains of PDW, and also provides some management functions for the HDI Region.

passive host

This server provides high availability. It is online and ready to run jobs in case there is a failure on the orchestration or data host. For the PDW Region, the PDW orchestration host, passive host, and Data Scale Unit servers are configured as a Windows failover cluster. Each rack in the appliance that contains PDW Data Scale Units requires one PDW passive host.

optional passive host

To add further redundancy, you have the option to add a second passive host to the PDW Base Scale Unit.

Data Scale Unit

The PDW Base Scale Unit includes one Data Scale Unit which is placed at the bottom of the rack.

This diagram shows the PDW Base Scale Unit plus the Rack and Network. This is the minimum configuration for an Analytics Platform System appliance.

PDW Base Scale Unit

[ TOP ]

HDI Base Scale Unit

In order to add an HDI Base Scale Unit, you must first have a PDW Base Scale Unit. The HDI Base Scale Unit contains the hosts and minimum storage that is required for an HDI Region. You can add more capacity to the HDI Region by adding more Data Scale Units.

orchestration host

This is a server that runs the brains of HDInsight, and it also provides some management functions for the HDI Region.

passive host

This is a passive server provided for high availability of the orchestration host and the data hosts. For the HDI Region, the HDI orchestration host, passive host, and Data Scale Unit servers are configured as a Windows failover cluster.

Data Scale Unit

The HDI Base Scale Unit contains one Data Scale Unit.

HDI Base Scale Unit

[ TOP ]


Hardware Configurations (Analytics Platform System)

[image]

Hardware Configurations (Analytics Platform System)
Send Feedback

SQL Server PDW hardware is architected with scalable units so that you buy the right amount of processing and storage according to your business requirements. The appliance scales storage for the SQL Server PDW Region from a few Terabytes to over 6 Petabytes of data.

Contents

  • PDW One-Rack Configurations

  • PDW Multi-Rack Configurations

  • HDI Minimum Configuration

PDW One-Rack Configurations

The first rack in the appliance contains the components required to run the PDW Region and the HDI Region, if there is an HDI Region. The PDW Region is required for all appliances, and so the minimum appliance configuration is a Rack and Network plus a PDW Base Scale Unit.

These diagrams show ways that the first rack of the appliance can be configured for the PDW Region. You can have between 2 and 9 Compute nodes in the first rack, depending on the hardware vendor.

First Rack Configurations - DELL

The minimum configuration for a DELL appliance has 3 Compute nodes. You can add up to 2 Data Scale Units to the first rack for a total of 9 Compute nodes.

Base Scale Unit

First Rack Configurations - HP

The minimum configuration for an HP appliance has 2 Compute nodes. You can add up to 3 Data Scale Units to the first rack for a total of 8 Compute nodes.

PDW first rack configurations for HP

PDW Multi-Rack Configurations

You add capacity to the PDW Region by adding Data Scale Units, along with additional Rack & Network components as necessary to provide the proper power, networking, and rack infrastructure. Each additional Rack & Network requires a PDW passive host.

Each hardware vendor specifies the number of Data Scale Units you can add given the capacity of your appliance. We recommend adding enough Data Scale units to see at least a 20 percent uplift in performance. For example, adding one Data Scale Unit to an appliance that already has 20 Data Scale Units might result in a negligible performance gain. The net gain wouldn’t be worth the cost and effort.

Scale Out Example - HP

This diagram shows a 3 rack HP appliance that contains 20 Compute nodes.

PDW Region with 20 Compute Nodes (HP)

Scale Out Example – DELL, Quanta

This diagram shows a 3 rack DELL or Quanta appliance that contains 21 Compute nodes.

Dell or Quanta PDW Region

HDI Minimum Configuration

To configure APS with an HDI Region, at a minimum you need the PDW Region plus the HDI Base Scale Unit. Data scale units in additional racks can also be dedicated to the HDI Region.

Note Note

When you determine the capacity of the HDI Region, please take into consideration that you cannot expand the HDI Region by simply adding more hardware. If you need to expand with this version, you can do this by adding hardware and then rebuilding the HDI Region.

Minimal HDI Region


PDW Processing and Storage Capacity (Analytics Platform System)

[image]

PDW Processing and Storage Capacity (Analytics Platform System)
Send Feedback

Your business requirements determine the number of Data Scale Units, and the size of the Compute node disks that you need in your SQL Server PDW appliance. Use these processing and storage calculations to guide your capacity purchasing and planning decisions.

Contents

  • Planning for Processing Capacity

  • Planning for Storage Capacity for PDW

Planning for Processing Capacity

PDW’s MPP query performance depends heavily on the number of CPU cores working on your data in parallel. Within limits, increasing parallelism improves MPP query performance. Even if your data size is relatively small, the power of the MPP query engine is enhanced by having greater parallelism.

For example, an appliance with 12 Compute nodes has 192 CPU cores that process your data in parallel. That’s 192-way parallelism! An appliance with 56 Compute nodes has 896 cores all working in parallel. This magnitude of parallelism is not achievable with SMP computing.

As the number of Compute nodes increases, scaling out the appliance requires adding more than one Compute node at a time to get a noticeable benefit. Hardware vendors support only specific configurations of Data Scale Units to ensure that the benefit of scaling the appliance outweighs the cost of redistributing the data across more Compute nodes.

Data Scale Unit Configuration Examples - HP

These are examples of the supported HP configurations for Data Scale Units. They might vary from the most current supported configurations, but are provided as an example of how to increase capacity by approximately 20 percent.

Uplift is the percent capacity gain by increasing the Data Scale Units from one row to the next. For example, increasing the Data Scale Units from 6 to 8 gives a 33% uplift in CPU cores and memory. It also increases the disk space which isn’t shown in this table.

Data Scale Units | Compute Nodes | CPU Cores | Memory (GB) | Uplift
1 | 2 | 32 | 512 | -
2 | 4 | 64 | 1024 | 100%
3 | 6 | 96 | 1536 | 50%
4 | 8 | 128 | 2048 | 33%
5 | 10 | 160 | 2560 | 25%
6 | 12 | 192 | 3072 | 20%
8 | 16 | 256 | 4096 | 33%
10 | 20 | 320 | 5120 | 25%
12 | 24 | 384 | 6144 | 20%
16 | 32 | 512 | 8192 | 33%
20 | 40 | 640 | 10240 | 25%
24 | 48 | 768 | 12288 | 20%
28 | 56 | 896 | 14336 | 17%

Explanation:

  • Data Scale Units per appliance. To learn about Data Scale Units, see Hardware Components (Analytics Platform System)

  • Compute Nodes per appliance.

  • CPU cores per appliance. There are 16 cores per Compute node, one core per each mirrored disk-pair. For Compute node disk structure, see Hardware Components (Analytics Platform System).

  • Memory per appliance. Each core has 256 GB memory.

Data Scale Unit Configuration Examples – Dell, Quanta

These are examples of the supported Dell and Quanta configurations for Data Scale Units. They might vary from the most current supported configurations, but are provided as an example of how to increase capacity by approximately 20 percent.

Uplift is the percent capacity gain by increasing the Data Scale Units from one row to the next. For example, increasing the Data Scale Units from 6 to 8 gives a 33% uplift in CPU cores and memory. It also increases the disk space which isn’t shown in this table.

Data Scale Units | Compute Nodes | CPU Cores | Memory (GB) | Uplift
1 | 3 | 48 | 768 | -
2 | 6 | 96 | 1536 | 100%
3 | 9 | 144 | 2,304 | 50%
4 | 12 | 192 | 3,072 | 33%
5 | 15 | 240 | 3,840 | 25%
6 | 18 | 288 | 4,608 | 20%
7 | 21 | 336 | 5,376 | 17%
8 | 24 | 384 | 6,144 | 14%
9 | 27 | 432 | 6,912 | 13%
12 | 36 | 576 | 9,216 | 33%
15 | 45 | 720 | 11,520 | 25%
18 | 54 | 864 | 13,824 | 20%

Planning for Storage Capacity

This table estimates that you could load and store up to 6 petabytes of uncompressed user SQL Server PDW data onto a fully built Analytics Platform System appliance. If the appliance has an HDInsight region, the maximum SQL Server PDW storage will be reduced according to the size of the HDInsight region.

Vendor | Drive Size | Physical Data Storage Per Compute Node | Maximum Compute Nodes Per Rack | Physical Maximum Data Storage Per Rack | Estimated Maximum User Data Storage Per Rack | Maximum Racks | Estimated Maximum User Data Storage Per Appliance
HP | 1 TB | 16 TB | 8 | 128 TB | 320 TB | 7 | 2,240 TB
HP | 2 TB | 32 TB | 8 | 256 TB | 640 TB | 7 | 4,480 TB
HP | 3 TB | 48 TB | 8 | 384 TB | 960 TB | 7 | 6,720 TB
DELL | 1 TB | 16 TB | 9 | 144 TB | 360 TB | 6 | 2,160 TB
DELL | 2 TB | 32 TB | 9 | 288 TB | 720 TB | 6 | 4,320 TB
DELL | 3 TB | 48 TB | 9 | 432 TB | 1080 TB | 6 | 6,480 TB

Explanation:

  • Drive Size is 1, 2, or 3 TB for each Hardware vendor.

  • Physical Data Storage Per Compute Node = (Drive Size) * (16 disks per Compute node). The mirrored disks are not included since they are for redundancy.

  • Maximum Compute Nodes Per Rack is specific to the hardware vendor.

  • Physical Maximum Data Storage Per Rack = (Physical data Storage Per Compute Node) * (Maximum Compute Nodes Per Rack).

  • Estimated Maximum User Data Storage Per Rack = (Physical Maximum Data Storage Per Rack) * (5 for a 5:1 compression ratio) * (50% for logs and tempDB). This is a conservative estimate for the uncompressed user data that can be loaded and stored onto the appliance. This is an estimate and is not enforced by software. The actual user data storage depends on your data and your configuration.

  • Maximum Racks is specific for each Hardware vendor.

  • Estimated Maximum Data Storage Per Appliance = (Estimated Maximum Data Storage Per Rack) * (Maximum Racks). This is a conservative estimate of the grand total size of user data that you could load and store on a fully built appliance.


PDW High Availability (Analytics Platform System)

[image]

PDW High Availability (Analytics Platform System)
Send Feedback

Describes how PDW and the Analytics Platform System appliance fabric are architected for high availability.

High Availability Architecture

Network

For network availability, the appliance has two InfiniBand networks. If one of the InfiniBand networks goes down, the other one is still available. Also, Active Directory has replicated domain controllers to resolve incoming requests to the correct InfiniBand network.

For more information, see Configure InfiniBand Network Adapters (SQL Server PDW).

Storage

To keep data safe, Analytics Platform System uses RAID 1 mirroring to maintain two copies of all user data. When a disk fails, the hardware system rebuilds the data onto a spare disk and sets an alert that there is a disk failure.

To keep data available online, Analytics Platform System uses Windows Storage Spaces and clustered shared volumes to manage the user data disks in the direct attached storage. There is one storage pool per Data Scale Unit organized into Cluster Shared Volumes which are available to the Compute node hosts through mount points.

To ensure the storage pool stays online, each host in the Data Scale Unit has an ISCSI virtual machine that does not failover. This architecture is important because if a host fails, the data is still accessible through the other host(s) in the Data Scale Unit.

Hosts

For host availability, all of the hosts are configured into a Windows Failover Cluster. Every rack has a passive host. Optionally the first rack, which controls PDW and the appliance fabric, can have a second passive host. If a host fails, virtual machines that are configured for failover, will fail over to an available passive host.

PDW Nodes and Appliance Fabric

For high availability of the PDW nodes and the appliance fabric, Analytics Platform System uses virtualization. Each of the PDW and appliance fabric components run in a virtual machine.

Each virtual machine is defined as a role in the Windows failover cluster. When a virtual machine fails, the cluster restarts it on an available passive host. The virtual machines are deployed using System Center Virtual Machine Manager. When a failover occurs, the virtual machine running on the passive host is still able to access its user data through the InfiniBand network.

More details, the Control node and Compute node virtual machines are each configured as a single-node cluster. The single-node cluster manages the InfiniBand networks as a cluster resource to ensure the cluster is always using the active InfiniBand IP. The single-node cluster manages the PDW processes that run within the virtual machine. For example, the single-node cluster has SQL Server and Data Movement Service (DMS) as resources so that it can start them in the proper order. The Control node VM also controls the start order for the other VMs that run on the orchestration host.


SQL Server Parallel Data Warehouse (SQL Server PDW)

[image]

SQL Server Parallel Data Warehouse (SQL Server PDW)
Send Feedback

SQL Server PDW runs as a workload on Analytics Platform System. Use SQL Server PDW as the core relational data warehousing component of your end-to-end business intelligence solutions.

SQL Server PDW Benefits

With PDW’s massively parallel processing (MPP) design, queries commonly complete 50 times faster than traditional data warehouses built on symmetric multi-processing (SMP) database management systems.

Note Note

50 times faster means that queries complete in minutes instead of hours, or seconds instead of minutes. With this breakthrough performance, your business analysts can generate more extensive results faster, and can easily perform ad-hoc queries or drill down into the details. As a result, your business can make better decisions, faster.

In addition to achieving breakthrough query performance, PDW makes it easy to:

  • Grow your data warehouse to anywhere from a few terabytes to over 6 petabytes of data in a single appliance by adding “scale units” to your existing system

  • Trust your data will be there when you need it because of the built- in high redundancy and high availability,

  • Solve modern data challenges of loading and consolidating data

  • Integrate Hadoop data with relational data for fast analysis by using PDW’s highly parallelized PolyBase technology

  • Use Business Intelligence tools to build comprehensive end-to-end solutions.

To learn more, see the whitepaper Parallel Data Warehouse: A Breakthrough Platform for Next-Generation Data Warehousing and Big Data Solutions on MSDN.

In this section

  • Understanding SQL Server PDW (SQL Server PDW)

  • Plan for Backup and Loading Hardware (SQL Server PDW)

  • Client Tools and Applications (SQL Server PDW)

  • Query (SQL Server PDW)

  • Load (SQL Server PDW)

  • Database Administration (SQL Server PDW)

  • SQL Reference (SQL Server PDW)

  • System Views (SQL Server PDW)

  • Glossary (SQL Server PDW)


Understanding SQL Server PDW (SQL Server PDW)

[image]

Understanding SQL Server PDW (SQL Server PDW)
Send Feedback

This topic explains the appliance software and the non-appliance software components of Analytics Platform System.

To learn more about Analytics Platform System, see:

  • APS Architecture (Analytics Platform System)

  • Distributed and Replicated Tables (SQL Server PDW)

  • PolyBase for Hadoop and WASB Integration (SQL Server PDW)

  • Clustered Columnstore Indexes (SQL Server PDW)

  • Query Process (SQL Server PDW)

  • Minimum and Maximum Values (SQL Server PDW)

Contents

  • Appliance Software

  • Solution Software and Tools

SQL Server PDW Software

Appliance Software – Query Processing and User Data Storage

Control Node

MPP Engine

The MPP Engine is the brains of the Massively Parallel Processing (MPP) system. It does the following:

  • Creates parallel query plans and coordinates parallel query execution on the Compute nodes.

  • Stores and coordinates metadata and configuration data for all of the databases.

  • Manages SQL Server PDW database authentication and authorization.

  • Tracks hardware and software status.

Data Movement Service (DMS)

Data Movement Service (DMS) is part of the “secret sauce” of PDW. It does the following:

  • Transfers data to and from the SQL Server PDW nodes.

  • Processes query operations that require transferring data among the nodes.

  • Improves query performance by optimizing data transfer speeds.

Admin Console

The Admin Console is a web application that presents the appliance state, health, and performance information.

Configuration Manager

The Configuration Manager (dwconfig.exe), is the tool that appliance administrators use to configure Analytics Platform System.

SQL Server Database

SQL Server manages all of the databases on the Control node.

  • The Shell database manages the metadata for all distributed user databases.

  • TempDB contains the metadata for all user temporary tables across the appliance.

  • Master is the master table for SQL Server on the Control node.

Compute Node

The Compute nodes are parallel data processing and storage units. They have direct attached storage and use SQL Server to manage user data.

Data Movement Service (DMS)

Data Movement Service (DMS) runs on each Compute node to do the following:

  • As part of processing parallel queries, DMS transfer data to and from other Computer nodes and the Control node.

  • DMS, running on each Compute node, receives data loads in parallel. Data is loaded in parallel directly from the loading server to the Compute nodes

  • DMS transfers data from each Compute node directly to the backup server.

  • Using PolyBase, DMS transfers data to and from an external Hadoop cluster, or the HDInsight Region on the appliance.

SQL Server Database

Each Compute node runs an instance of SQL Server to process queries and manage user data.

Appliance Fabric

The appliance fabric provides the operating system, services, and network infrastructure for the appliance.

Domain Controller

Active Directory (AD) Domain Services (DS)

Analytics Platform System performs authentication among the Analytics Platform System nodes, and manages the authentication of SQL Server PDW Windows Authentication logins.

DNS service

Windows Domain Name Service (DNS) resolves domain names to IP addresses for the Analytics Platform System appliance.

Windows Deployment Service

Windows Deployment Service (WDS) deploys the Windows Server operating system onto the appliance. It is deployed on every host and virtual machine across the appliance.

The DHCP service creates IP addresses so that the hosts within the appliance domain can join the appliance network without having a pre-configured IP address.

Virtual Machine Manager

Analytics Platform System uses virtualization to achieve high availability. The Virtual Machine Manager hosts System Center to deploy the operating system on the physical hosts.

Windows Server Update Services (WSUS) to apply or remove Windows Updates across all of the hosts and virtual machines.

Windows Server

All of the hosts and virtual machines in the appliance run Windows Server operating system.

Failover Clustering

Windows Failover Clustering provides the ability to restart processes on a passive host in the event that a host fails.

Storage Spaces

Windows Storage Spaces manages user data as a storage pool for a small group of Compute nodes. If a Compute node fails, the data is still accessible through another Compute node in the group.

Hyper-V

Microsoft Hyper-V Server provides a simple and reliable virtualization solution. Analytics Platform System uses virtualizations to balance CPU resources and to provide high availability for the PDW nodes and appliance fabric components .

Solution Software and Tools

Non-Relational Data

PolyBase technology to integrates SQL Server PDW data with external Hadoop data. The Hadoop data can be stored on any of these Hadoop Data sources:

  • Hortonworks for Linux or Windows Server

  • Cloudera on Linus

  • HDInsight running on APS

  • HDInsight data stored on Azure Storage Blob

Microsoft Client Tools

Query Tools

Queries are written with Transact-SQL modified to fit the MPP nature of the queries. All queries are submitted to the Control node which generates a parallel query plan to run the query across the Compute nodes.

SQL Server database tooling

SQL Server database tooling (formerly called SQL Server Data Tools) runs inside of Visual Studio and is our recommended GUI tool for submitting queries to SQL Server PDW. It is similar to SQL Server Management Studio by allowing you to navigate through an object explorer.

If you don’t already have Visual Studio, you can download the tools that you need for free. For more information, see Install SQL Server database tooling for Visual Studio (SQL Server PDW).

sqlcmd Command-Line Query Tool

sqlcmd is the SQL Server command-line tool for running Transact-SQL statements and system commands. It works with SQL Server PDW and is our recommended command-line tool for querying SQL Server PDW.With sqlcmd you can run Transact-SQL statements interactively from the command-line, as a batch file, or from Windows PowerShell.

If you don’t have SQL Server, you can download this as a standalone package. For more information, see Install sqlcmd Command-Line Client (SQL Server PDW)

Integration Services

You can use Integration Services to query SQL Server PDW. For more information, see Connect With SQL Server Integration Services for Querying (SQL Server PDW).

Linked Server

By using a SQL Server linked server connection, you can use SQL Server to submit Transact-SQL statements to SQL Server PDW. For more information, see Connect With SQL Server Linked Server (SQL Server PDW).

Business Intelligence Tools

Analysis Services

SQL Server PDW is a valid data source for Analysis Services databases and Excel PowerPivot models. Using the OLE DB provider, you can configure an Analysis Services cube to use either multidimensional online analytical processing (MOLAP) or relational online analytical processing (ROLAP) storage.

For more information, see Connect With SQL Server Analysis Services (SQL Server PDW).

Report Builder

You can use SQL Server PDW as a SQL Server data source for reports that you develop for Reporting Services by using SQL Server Report Builder. You can also use SQL Server PDW as a SQL Server source for report models. By using Report Manager or the report server API, you can generate a model from a SQL Server PDW database.

For more information, see Connect With SQL Server Report Builder (SQL Server PDW) and Connect With SQL Server Reporting Services (SQL Server PDW).

Power Pivot for Excel

You can connect to SQL Server PDW with PowerPivot for Excel, a free download that significantly expands the data analysis capabilities of Excel.

For more information, see Connect With PowerPivot for Excel (SQL Server PDW).

Loading Tools

Integration Services

  • Install PDW-specific destination adapters that allow you to use SQL Server Integration Services to load data into SQL Server PDW.

    For more information, see Install Integration Services Destination Adapters (SQL Server PDW).

dwloader Command-Line Loader

dwloader is a command-line loading tool that loads data in parallel from your loading server to the SQL Server PDW Compute nodes. For more information, see Install dwloader Command-Line Loader (SQL Server PDW)

PolyBase for Hadoop Integration

With PolyBase technology, you can load non-relational data from an Hadoop Cluster into a relational table in SQL Server PDW. The Hadoop data can be located in an external Hadoop Cluster, the HDI Region on APS, or in a Windows Azure Storage Blob.

For more information, see PolyBase for Hadoop and WASB Integration (SQL Server PDW).

Database Backup and Restore

SQL Server PDW uses Transact-SQL database backup and restore commands to backup and restore user databases, in parallel, to and from a backup server. SQL Server PDW writes the backup to a directory in a Windows file share, and then likewise restores data from a Windows file share.

For more information, see Plan for Backup and Loading Hardware (SQL Server PDW) and Backup and Restore (SQL Server PDW)

Remote Table Copy

The Remote Table Copy feature allows you to copy tables from SQL Server PDW databases to remote (non-appliance) SMP SQL Server databases. This enables hub and spoke scenarios for SQL Server PDW.

For more information, see Remote Table Copy (SQL Server PDW).

Monitoring

Analytics Platform System has several ways to monitor appliance activity

Admin Console

The Admin Console allows you to view current status about the appliance health. This runs as a web application on the Control node and is accessible over https.

For more information, see Monitor the Appliance by Using the Admin Console (Analytics Platform System)

System Views

The Admin Console is based on system view queries. You can query the system views individually to get the specific piece of information you need.

For more information, see Monitor the Appliance by Using System Views (Analytics Platform System)

System Center Operations Manager

There are System Center Operations Manager (SCOM) Management Packs for SQL Server PDW. To configure the appliance for SCOM, see Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)


Distributed and Replicated Tables (SQL Server PDW)

[image]

Distributed and Replicated Tables (SQL Server PDW)
Send Feedback

Describes distributed tables and replicated table concepts and provides links to database design best practices for SQL Server PDW databases. Designing SQL Server PDW databases to use distributed and replicated tables effectively will help you to achieve the storage and query processing benefits of the MPP architecture. A small number of database design choices can have a big impact on improving query performance.

Contents

  • Understanding Distributed Tables

  • Understanding Replicated Tables

  • Best Practices

Understanding Distributed Tables

Distributed table

A distributed table is a table whose rows are dispersed across multiple storage locations. Each storage location contains a group of one or more rows, called a distribution.

The following diagram depicts how a full (non-distributed table) gets stored as a distributed table.

  • The distributions are stored on the Compute nodes; the appliance has eight distributions per compute node.

  • Each row belongs to one and only one distribution.

  • SQL Server PDW uses a deterministic hash algorithm to assign each row to a distribution.

  • The number of table rows per distribution can vary as shown by the different sizes of tables.

Distribution

The set of table rows, from one or more distributed tables that are stored together in the same storage location. To improve parallel query performance, the rows that will be joined together in queries should be stored together in the same distributions; this avoids transferring data among the distributions before running parallel query operations.

A deterministic function assigns each row to belong to one and only one distribution. SQL Server PDW uses the value in the distribution column to assign each row to a distribution.

To embrace the shared nothing architecture, each distribution is stored on its own set of disks.

In SQL Server PDW, each Compute node has eight distributions. For example, if there are eight Compute nodes, there will be 64 distributions (8 nodes x 8 distributions/node).

Distribution column

The column that SQL Server PDW uses to assign a distributed table row to a distribution. Each distributed table has one column designated as the distribution column.

There are performance considerations for the selection of a distribution column, such as distinctness, data skew, and the types of queries run on the system.

Data skew

Data skew is a condition that occurs when the rows of a distributed table are not spread uniformly across all of the distributions. When the number of rows in each distribution varies, this can negatively impact query performance.

Avoiding data skew means you want to avoid having a disproportionately large number of table rows in one distribution in comparison to the number of rows in other distributions. Since queries rely on all distributions, even if queries finish fast in smaller distributions, you will still need to wait for the queries to finish on all distributions before you can get the query result.

Note Note

A parallel query performs as slow as the ‘slowest’ distribution.

[TOP]

Understanding Replicated Tables

Replicated table

A replicated table is a table that is stored in its entirety on each Compute node. Replicating a table removes the need to transfer its table rows among Compute nodes before performing a JOIN operation. Replicated tables are only feasible with small tables because of the extra storage required to store the full table on each compute node.

The following diagram shows a replicated table that is stored on each Compute node. The replicated table is stored across all of the disks assigned to the Compute node. This is implemented by using SQL Server filegroups.

Replicated Table Stored on All Compute Nodes

[TOP]

Best Practices

For more information about best practices for designing SQL Server PDW tables, see the following wiki articles on Technet. If you cannot access these links, see Join Our Communities (Analytics Platform System)

  • Choosing Distributed Versus Replicated Tables in SQL Server PDW

  • Minimize Data Movement When Choosing a Distribution Column (SQL Server PDW)

  • Minimize Data Skew When Choosing a Distribution Column (SQL Server PDW)

  • None of the Existing Columns Are a Good Choice for the Distribution Column (SQL Server PDW)


PolyBase for Hadoop and WASB Integration (SQL Server PDW)

[image]

PolyBase for Hadoop and WASB Integration (SQL Server PDW)
See Also Send Feedback

Describes PolyBase for Hadoop Integration with SQL Server PDW.

PolyBase is a fundamental breakthrough in data processing used in SQL Server PDW. It enables truly integrated querying across Hadoop non-relational data, Windows Azure blob storage (WASB) data, and SQL Server PDW relational data in a fully parallelized fashion. With Polybase you can query data stored in these external data sources by using standard SQL statements without learning MapReduce techniques.

Before You Begin

Software Prerequisites

If not already configured, ask your appliance administrator to configure the appliance for Hadoop connectivity. For configuration instructions, see Configure PolyBase Connectivity to External Data (Analytics Platform System). For a list of supported Hadoop versions, see sp_configure (SQL Server PDW).

Basics

Key Terms

The following are key terms and concepts that you will need to know in order to better understand how to move data between a Hadoop Cluster and SQL Server PDW.

Hadoop Distributed File System (HDFS)

The Hadoop Distributed File System (HDFS) is the file system that Hadoop uses to store nonrelational data.

external table

Refers to data that resides outside of the appliance. The table definition is stored in the SQL Server PDW metadata. In this release, delimited text files are the only supported file format for accessing data that is stored in external tables.

PolyBase for Hadoop Integration uses external tables to import Hadoop data into SQL Server PDW, query Hadoop data from SQL Server PDW, and export data from SQL Server PDW to Hadoop.

PolyBase Configuration

Beginning with APS AU2, you can set a polybase.pushdown.dfs.replication parameter in either the core-site.xml or hdfs-site.xml files on the Control node. This performance related setting defines the data replication ratio for pushdown output. When set, it takes integer values between 1 and the maximum replication value allowed by the target cluster. If not set, this value defaults to 1. When set to a larger value, it may require more time for mappers to finish writing results to HDFS. The setting has the same effect whether set in core-site or hdfs-site. If set in both and core-site is not marked as final, the setting in hdfs-site is used. (This is the standard behavior of Hadoop.) This property only applies to data inserted by using Polybase.

The Hadoop configuration files (such as yarn-site.xml and core-site.xml) used by PolyBase are stored on the Control Node at C:\program files\Microsoft SQL Server Parallel Data Warehouse\100\Hadoop\conf\. You add or must modify the yarn.application.classpath configuration in the yarn-site.xml file to provide the APS Hadoop file location values specific to your environment. The yarn.application.classpath configuration setting in the yarn-site.xml files must be identical on both APS (for PolyBase) and on the Hadoop target that PolyBase is connecting to.

Related Statements

A. CREATE EXTERNAL DATA SOURCE

CREATE EXTERNAL DATA SOURCE (SQL Server PDW) creates an external data source. This objects contains the location of the data and an optional Hadoop job tracker ID. For a Hadoop cluster, the location is a Hadoop Distributed File System (HDFS) URL (e.g. 'hdfs://ip_address:port'). For WASB, the location is the WASB container and path (e.g., 'wasbs://[ container@ ] account_name.blob.core.windows.net/path').

B. CREATE EXTERNAL FILE FORMAT

CREATE EXTERNAL FILE FORMAT (SQL Server PDW) creates a file format definition for data stored in an external fileHadoop.

c. CREATE EXTERNAL TABLE

CREATE EXTERNAL TABLE (SQL Server PDW) creates the external table definition and stores the definition in SQL Server PDW metadata. The external table definition points to text-delimited files that reside on a Hadoop Cluster. The data in the Hadoop text-delimited files is populated by your own Hadoop process.

Use CREATE EXTERNAL TABLE to define an external table so that you can run SQL Server PDW SQL queries to select or import Hadoop data.

D. CREATE TABLE AS SELECT

CREATE TABLE AS SELECT (SQL Server PDW) creates a new table in SQL Server PDW that is populated with the results of a SELECT statement. By referencing an external table in the SELECT FROM clause, you can import data from Hadoop for persistent storage in either a distributed or replicated SQL Server PDW table.

To import Hadoop data from an external table, use CREATE TABLE AS SELECT and select from the external table.

C. CREATE EXTERNAL TABLE AS SELECT

CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW) creates a new external table, and exports the results of the SELECT statement to Hadoop. The data are stored in text-delimited files in Hadoop. The location of the files is specified in the table definition. exports the results of the SELECT statement to the text-delimited files on a Hadoop cluster.

Use CREATE EXTERNAL TABLE AS SELECT to export data from SQL Server PDW to Hadoop where the data will be stored as text-delimited files.

D. DROP EXTERNAL TABLE

DROP TABLE (SQL Server PDW) includes the DROP EXTERNAL TABLE statement. This drops the external table. Dropping an external table removes the table definition from SQL Server PDW. It does not remove any data that exists on the Hadoop.

Related Tasks

Task | Task Description
Configure the appliance for Hadoop connections. | Configure PolyBase Connectivity to External Data (Analytics Platform System)
Create an external table for the purpose of importing data from Hadoop into SQL Server PDW. | CREATE EXTERNAL TABLE (SQL Server PDW)
Query Hadoop data, stored in an external table, from SQL Server PDW. | SELECT (SQL Server PDW), FROM (SQL Server PDW) Specify the name of the external table as the table source in the FROM clause of a SELECT statement.
Import data from Hadoop and store it into a SQL Server PDW table. | CREATE TABLE AS SELECT (SQL Server PDW) Use CREATE TABLE AS SELECT to create a new table in SQL Server PDW and populate it with the results of a SELECT statement by specifying the name of the external table as the table source in the FROM clause.
Export data from SQL Server PDW and save it in Hadoop. | CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW) Use CREATE EXTERNAL TABLE AS SELECT to create a new external table, and use the SELECT statement to define which data you want to export to Hadoop. SQL Server PDW will export the results of the SELECT statement to the external table stored on Hadoop.

See Also

Concepts

Understanding SQL Server PDW (SQL Server PDW)

Other Resources

PolyBase


Clustered Columnstore Indexes (SQL Server PDW)

[image]

Clustered Columnstore Indexes (SQL Server PDW)
Send Feedback

The in-memory clustered columnstore index stores and manages data by using column-based data storage and column-based query processing. Use this storage method to achieve up to 10x query performance gains over traditional row-oriented storage, and up to 7x data compression over the uncompressed data size.

Note Note

We view the clustered columnstore index as the standard for storing tables in SQL Server PDW, and expect it will be used in most scenarios.

The clustered columnstore index:

  • Is updateable.

  • Includes all columns of the table and is the means for which the entire table is stored.

  • Delivers high query performance by storing and compressing data by columns.

  • Benefits from high compression ratios by using fewer reads to bring compressed data into memory and then using the reduced data volume for the in-memory processing.

Contents

  • Columnstore Basics

  • Columnstore Processes

  • Related Statements

  • Metadata

  • Performance Tips

  • Related Tasks Examples

Basics

Clustered Columnstore Index

A clustered columnstore index is a technology for storing, retrieving and managing data by using a columnar data format, called a columnstore. The data is compressed, stored, and managed as a collection of partial columns, called column segments.

Some of the clustered columnstore index data is stored temporarily in a rowstore table, called a deltastore, until it is compressed and moved into the columnstore. The clustered columnstore index operates on both the columnstore and the deltastore, to return the correct query results.

The remainder of this topic explains these concepts and provides examples.

[ TOP ]

Key Characteristics

In SQL Server PDW, a clustered columnstore index:

  • Includes all columns in the table. It is the method for storing the entire table.

  • Is the only index on the table. It cannot be combined with other indexes.

  • Uses columnstore compression. The compression is not configurable.

  • Does not physically store columns in a sorted order. Instead, it stores data to improve compression and performance..

  • Applies after the user data is copied to the correct Compute node, distribution, and partition. SQL Server PDW moves the data to the correct location, and then adds the data to the clustered columnstore index.

  • Is comprised of many parallel clustered columnstore indexes. For a distributed table, there is one clustered columnstore index for every partition of every distribution and for every Compute node. For a replicated table, there is one clustered columnstore index for every partition of the replicated table on every Compute node. .

[ TOP ]

Benefits

SQL Server takes advantage of the column-based data layout to significantly improve compression rates and query execution time. For example:

  • Columns often have similar data, which results in high compression rates. In turn, higher compression rates further improve query performance by using a smaller in-memory footprint.

  • High compression rates improve query performance by using a smaller in-memory footprint. In turn, query performance can improve because SQL Server PDW can perform more query and data operations in-memory.

  • Queries often select only a few columns from a table, which reduces total I/O to and from the physical media.

  • Advanced query execution technology processes chunks of columns called batches in a streamlined manner, which reduces CPU usage.

[ TOP ]

Key Terms

The following are key terms and concepts that you will need to know in order to better understand how to use clustered columnstore indexes.

rowgroup

A rowgroup is a group of rows that are compressed into columnstore format at the same time. Each column in the rowgroup is compressed and stored separately onto the physical media. For the minimum and maximum rows per rowgroup, see Minimum and Maximum Values (SQL Server PDW).

column segment

A column segment is the basic storage unit for a clustered columnstore index. It is a group of column values that are compressed and physically stored together on the physical media. Rowgroups define the column values that are in each column segment.

  1. Each column is comprised of one or many column segments.

  2. Each rowgroup contains one column segment for every column in the table.

  3. When SQL Server PDW compresses a rowgroup, it compresses each column within the rowgroup as one column segment.

columnstore

A columnstore is data that is logically organized as a table with rows and columns, and physically stored in a columnar data format. In SQL Server PDW, the columnstore contains compressed column segments.

rowstore

A rowstore is data that is organized as rows and columns, and then physically stored in a row-wise data format. This has been the traditional way to store relational table data.

deltastore

A deltastore is a rowstore table that holds rows until the number of rows is large enough to be moved into the columnstore. During a bulk load, most of the rows go directly to the columnstore without passing through the deltastore.

When you perform a bulk load, most of the rows will go directly to the columnstore without passing through the deltastore. Some rows at the end of the bulk load might be too few in number to meet the minimum size of a rowgroup. When this happens, the final rows go to the deltastore instead of the columnstore.

Rows accumulate in each deltastore until the number of rows is the maximum number of rows allowed for a rowgroup.

When a deltastore contains the maximum number of rows per rowgroup, SQL Server PDW marks the rowgroup as “CLOSED”. A background process, called the “tuple-mover”, then moves the rowgroup into the columnstore. The rowgroup is marked as “COMPRESSED”.

When the deltastore contains the maximum number of rows per rowgroup, SQL Server PDW marks the rowgroup as “CLOSED”. A background process, called the “tuple-mover”, finds the CLOSED rowgroup and moves into the columnstore. The rowgrop is compressed into column segments and the column segments are stored in the columnstore.

For each columnstore there can be multiple deltastores.

  • If a deltastore is locked, SQL Server PDW will try to obtain a lock on a different deltastore. If there are no deltastores available, SQL Server PDW will create a new deltastore.

  • For a distributed table, there are one or more deltastores for every partition of every distribution across all the Compute nodes..

  • For a replicated table, there are one or more deltastores for each partition for each replicated table across all of the Compute nodes.

[ TOP ]

Processes

Create a Clustered Columnstore Index

There are multiple ways to create a clustered columnstore index. For example:

  • Use CREATE TABLE (SQL Server PDW) and the CLUSTERED COLUMNSTORE INDEX option to create a new table.

  • Use CREATE TABLE AS SELECT (SQL Server PDW) and the CLUSTERED COLUMNSTORE INDEX option to create a new table based on copying data from an existing table. For best performance, we recommend this method for converting large fact tables from rowstore to columnstore.

  • Use CREATE COLUMNSTORE INDEX (SQL Server PDW) to convert a rowstore table to a columnstore table. The syntax for this approach is simple and easy to use. This approach is performed as an atomic operation within each Compute node. It runs serially across each Compute node, whereas CREATE TABLE AS SELECT is perfomed in parallel across the distributions and across the Compute nodes. For best performance, we recommend this approach to save syntax development time if you need to convert many small dimension tables from rowstore to columnstore.

See Examples.

Add Data to a Clustered Columnstore Index

You can add data to an existing clustered columnstore index by using any of the standard loading methods. For example, dwloader, Integration Services, and INSERT … SELECT can all load data into a clustered columnstore index. For more information about loading methods, see Load (SQL Server PDW).

To load data into a clustered columnstore index, SQL Server PDW first moves the data the correct physical location on the appliance.

  • For a partitioned, distributed table, the location is a partition within a distribution on a Compute node.

  • For a non-partitioned, distributed table, the location is a distribution on a Compute node.

  • For a replicated table, the location is each Compute node.

The following diagram shows how data gets added to a clustered columnstore index. This operation occurs per partition, and per distribution for a distributed table, and per Compute node for a replicated table.

Loading into a clustered columnstore index

As the diagram suggests, to load data into a clustered columnstore index, SQL Server:

  1. Divides the bulk load into rowgroups of the maximum 1,048,576 rows.

    1. For distributed tables, the rowgroups are at the partition level. For example, a distributed table with 10 logical partitions on a 6 Compute node appliance with 8 distributions per Compute node, will be stored with 480 actual partitions. Rows will be loaded into a minimum of 480 rowgroups and 480 deltastores.

    2. For replicated tables, the rowgroups are per partition cross all of the Compute nodes. For example, a replicated table with 10 logical partitions on a 6 Compute node appliance will be stored with 60 actual partitions since the replicated table is duplicated on each Compute node.

  2. For each maximum-size rowgroup, SQL Server PDW will:

    1. Mark the rowgroup as CLOSED.

    2. Bypass the deltastore.

    3. Compress each column segment with the rowgroup with columnstore compression.

    4. Physically store each compressed column segment into the columnstore.

  3. Inserts the final rows of the bulk load as follows:

    1. If the number of rows meets the 102,400 minimum rows per rowgroup requirement, the rowgroup is compressed into the columnstore.

    2. If the number of rows is less than the 102,400 minimum rows per rowgroup, the rows are added to the deltastore.

Columnstore compression uses in-memory operations to compress each rowgroup. If all rows in a specific rowgroup (i.e. 1,048,576 rows) cannot fit into the available memory, the compression routines will reduce the number of rows in that rowgroup. As a result, multiple smaller compressed rowgroups could be added to the columnstore index. As rowgroups get smaller, the resulting columnstore compression degrades.

Columnstore Versus Deltastore Loading Scenarios

The following examples describe when loaded rows go directly to the columnstore or when they go to the deltastore. The minimum number of rows per rowgroup is 102,400, and the maximum number of rows per rowgroup is 1,048,576.

Rows to Bulk Load | Rows Added to the Columnstore | Rows Added to the Deltastore
102,000 | 0 | 102,000
145,000 | 145,000 Rowgroup size: 145,000 | 0
1,048,577 | 1,048,576 Rowgroup size: 1,048,576. | 1
2,252,152 | 2,252,152 Rowgroup sizes: 1,048,576, 1,048,576, 155,000. | 0

Loading Example

The following example shows the results of loading 1,048,577 rows into a partition. The results show that one COMPRESSED rowgroup in the columnstore (as compressed column segments), and 1 row in the deltastore.

Copy image Copy Code
SELECT * FROM sys. pdw_nodes_column_store_row_groups

Rowgroup and deltastore for a batch load

[ TOP ]

Perform DML Operations on a Clustered Columnstore Index

Clustered columnstore indexes support the INSERT, UPDATE, DELETE DML options.

INSERT

To insert a row with the INSERT statement:

  • SQL Server PDW inserts the new row into the deltastore.

DELETE

To delete a row with the DELETE statement:

  • If the row is in the columnstore, SQL Server PDW marks the row as logically deleted but does not reclaim the physical storage for the row until the index is rebuilt.

  • If the row is in the deltastore, SQL Server PDW logically and physically deletes the row.

UPDATE

To update a row with the UPDATE statement:

  • If the row is in the columnstore, SQL Server PDW marks the row as logically deleted, and then inserts the updated row into the deltastore.

  • If the row is in the deltastore, SQL Server PDW updates the row in the deltastore.

[ TOP ]

Rebuild a Clustered Columnstore Index

Rebuilding a clustered columnstore index removes fragmentation by re-compressing all of the data in the index. It also forces all of the data to be stored in the columnstore. The deltastore is empty after a rebuild.

Note Note

Since rebuilding forces all of the data into the columnstore, some of the rowgroups might have less than the 102,400 minimum rows per rowgroup that is used for bulk loading.

You can use CREATE COLUMNSTORE INDEX or ALTER INDEX to perform a full rebuild of an existing clustered columnstore index. Additionally, you can use ALTER INDEX to rebuild a specific partition.

To rebuild a clustered columnstore index, SQL Server:

  • Acquires an exclusive lock on the table or partition while the rebuild occurs. The data is “offline” and unavailable during the rebuild.

  • Defragments the columnstore by physically deleting rows that have been logically deleted from the table; the deleted bytes are reclaimed on the physical media.

  • Merges the rowstore data, stored in the deltastore, with the data in the columnstore before it rebuilds the index. When the rebuild is finished, all data is stored in columnstore format, and the deltastore, which is a rowstore, contains 0 rows.

  • Re-compresses all data into the columnstore. Two copies of the columnstore index exist while the rebuild is taking place. When the rebuild is finished, SQL Server deletes the original columnstore index.

When to rebuild a clustered columnstore index:

  • Use caution before rebuilding an entire clustered columnstore index because this takes a long time if the index is large, and it requires enough disk space to store an additional copy of the index while it is rebuilding. Usually it is only necessary to rebuild the most recently used partition

  • Rebuild a partition after heavy DML operations to defragment the index and reduce disk storage. For example, deleting rows marks them for deletion but does not actually delete the row from the columnstore until the index is rebuilt. Delting rows also occurs during an update operation; to update a row, SQL Server PDW inserts a new row, and then marks the old row for deletion.

  • Rebuild a partition after loading data to ensure all data is stored in the columnstore. This is useful, especially if the loaded rows land in one or more deltastores of the individual distributions. Note that if multiple loads occur at the same time, each distribution could end up having multiple deltastores.

[ TOP ]

Reorganize a Clustered Columnstore Index

You can use ALTER INDEX … REORGANIZE to move all CLOSED rowgroups into the columnstore.

When to reorganize a clustered columnstore index:

  • Reorganize a clustered columnstore index after one or more data loads to achieve query performance benefits as quickly as possible. Reorganizing will initially require additional CPU resources to compress the data, which could slow overall system performance. However, as soon as the data is compressed, query performance can improve.

Reorganizing is not required for moving CLOSED rowgroups into the columnstore. The tuple-mover process will eventually find all CLOSED rowgroups and move them. However, the tuple-mover is single-threaded and might not move rowgroups fast enough for your workload.

Related Statements

A. CREATE TABLE

Use the CREATE TABLE (SQL Server PDW) statement and CLUSTERED COLUMNSTORE INDEX table option to create a new columnstore table with a clustered columnstore index.

[ TOP ]

B. CREATE COLUMNSTORE INDEX

Use the CREATE COLUMNSTORE INDEX (SQL Server PDW) statement to rebuild an existing clustered columnstore index, or to convert a small dimension table from a rowstore to a columnstore.

Using CREATE COLUMNSTORE INDEX to convert a rowstore table to a columnstore table:

  • Retains the metadata and objects associated with the table. For example, the columnstore table retains security permissions, default constraints, views, and statistics.

  • Is simple and convenient to use, especially if you need to convert a large number of small dimension tables.

  • Should only be used for small dimension tables because of performance considerations. For more information, see the Performnace section in this topic.

For both rebuilding and converting operations, you can keep the same index name by using the DROP_EXISTING option.

[ TOP ]

C. CREATE TABLE AS SELECT

Use the CREATE TABLE AS SELECT (SQL Server PDW) statement to:

  • Convert a large rowstore table to a columnstore table. You can quickly start using the new table while the conversion occurs in parallel across all dimensions, and in the background.

  • Or, to create an additional clustered columnstore index that includes only a few of the columns in your original table. This requires extra storage space since data is copied to the new table.

Using CREATE TABLE AS SELECT to convert a rowstore table to a columnstore table:

  • Does not remove the original table. You need to drop the original table, if desired, after CREATE TABLE AS SELECT finishes.

  • Copies the table data to a temporary rowstore table and make the index available for read and write operations before all data has been moved to the columnar data format.

  • Performs the conversions from rowstore to columnstore in parallel across the distributions.

[ TOP ]

D. DROP INDEX

Use the DROP INDEX (SQL Server PDW) statement to drop a clustered columnstore index. This operation will drop the index and convert the columnstore table to a rowstore heap.

[ TOP ]

E. ALTER INDEX

Use the ALTER INDEX (SQL Server PDW) statement to rebuild or reorganize an existing columnstore index.

Using ALTER INDEX, the rebuild:

[ TOP ]

Metadata

All of the columns in a clustered columnstore index are stored in the metadata as included columns. The clustered columnstore index does not have key columns.

  • sys.indexes (SQL Server PDW)

  • sys.index_columns (SQL Server PDW)

  • sys.partitions (SQL Server PDW)

  • sys.pdw_nodes_column_store_segments (SQL Server PDW)

  • sys.pdw_nodes_column_store_dictionaries (SQL Server PDW)

  • sys.pdw_nodes_column_store_row_groups (SQL Server PDW)

[ TOP ]

Performance Tips

A. Use CREATE TABLE AS SELECT to convert large tables from rowstore to columnstore.

For best performance, use CREATE TABLE AS SELECT instead of CREATE COLUMNSTORE INDEX to convert fact and large dimension tables from rowstore to columnstore. The CREATE COLUMNSTORE INDEX operation is an easy way to convert small dimension tables; it is a slow way to convert large tables. For example:

  • The CREATE CLUSTERED COLUMNSTORE INDEX operation is atomic. To guarantee that all changes for a distributed table can be reversed in case of failure, SQL Server PDW creates a global transaction, distributed across all Compute nodes, that creates the clustered columnstore indexes serially across the distributions within each Compute node, and in parallel across all of the Compute nodes. This is several times slower than the CTAS statement, which does not run as a global distributed transaction, and therefore runs in parallel across the distributions on each Compute node.

B. Convert a fact table to a columnstore table before creating additional small columnstore tables based on the fact table.

If you plan to create smaller tables from a rowstore fact table, convert the fact table to a columnstore table with CREATE TABLE AS SELECT, and then create the smaller tables. This method is much faster than creating smaller tables from a rowstore fact tables. When the table is already stored aross the distributions with a clustered columnstore index, the data is already stored and compressed as columns. This makes it simple to create a new columnstore table.

Conversely, if the original table is a rowstore, SQL Server PDW will have to convert the data from rowstore to columnstore for each new small table. This takes more time than converting from columnstore to columnstore.

C. Use ALTER INDEX REBUILD to rebuild only a specific partition of the columnstore table.

Large fact and large dimension tables are usually partitioned in order to perform backup and management operations on chunks of the table. For partitioned tables, you do not need to rebuild the entire clustered columnstore index because fragmentation is likely to occur in only the partitions that have been modified recently. This is much more practical than rebuilding the entire table.

[ TOP ]

Examples

A. Create a New “Empty” Columnstore Table

To create a new columnstore table as an empty table, use the CREATE TABLE statement with the CLUSTERED COLUMNSTORE INDEX option. Columnstore tables support the standard table options such as distributed, replicated, and partitioned. They also support the standard table arguments such as default constraints, and collations.

The clustered columnstore index does not affect how the data is distributed to the Compute nodes. Instead, it affects the way the data is stored after it is distributed; the rows within each distribution are stored together as a columnstore. Similarly, replicated tables are stored as columnstore tables on each Compute node.

The following example creates a distributed table with a clustered columnstore index.

Copy image Copy Code
--Create a distributed table with a clustered columnstore index.
CREATE TABLE DistFactTable (
    colA int CONSTRAINT constraint_colA DEFAULT 0,
    colB nvarchar(32) COLLATE Frisian_100_CS_AS
)
WITH (
    DISTRIBUTION = HASH ( colB ),
    CLUSTERED COLUMNSTORE INDEX
);

The following example creates a replicated table with a clustered columnstore index.

Copy image Copy Code
--Create a replicated dimension table with a clustered columnstore index.
CREATE TABLE RepDimTable (
    id int NOT NULL,
    lastName varchar(20),
    zipCode varchar(6)
)
WITH (
    DISTRIBUTION = REPLICATE,
    CLUSTERED COLUMNSTORE INDEX
);

B. Convert a small dimension table from rowstore to columnstore

This example uses CREATE COLUMNSTORE INDEX (SQL Server PDW) to convert a small dimension table from a rowstore table to a columnstore table with a clustered columnstore index. This conversion method is a simple way to convert small dimension tables; it is a slow way to convert large tables.

Important note Important

For large tables, use CREATE TABLE AS SELECT instead of CREATE COLUMNSTORE INDEX to convert fact tables and large dimension tables from rowstore to columnstore.

To convert a small replicated rowstore table to a columnstore table.

  1. Before you begin, create a small dimension table to use in the example.

    Copy image Copy Code
    --Create a rowstore dimension table with a clustered index and also a non-clustered index.
CREATE TABLE MyDimTable (
    ProductKey [int] NOT NULL,
    OrderDateKey [int] NOT NULL,
     DueDateKey [int] NOT NULL,
     ShipDateKey [int] NOT NULL )
)
WITH (
    DISTRIBUTION = REPLICATE,
    CLUSTERED INDEX ( ProductKey )
);

--Add a non-clustered index.
CREATE INDEX my_index ON MyDimTable ( ProductKey, OrderDateKey );

  2. Drop all non-clustered indexes from the rowstore table.

    Copy image Copy Code
    --Drop all non-clustered indexes
DROP INDEX my_index ON MyDimTable;

  3. Drop the clustered index.

    • Do this only if you want to specify a new name for the index when it is converted to a clustered columnstore index. If you do not drop the clustered index, the new clustered columnstore index will have keep the same name.

      Note Note

      The name of the index might be easier to remember if you use your own name. All rowstore clustered indexes use the default name which is 'ClusteredIndex_<GUID>'.

    Copy image Copy Code
    --Process for dropping a clustered index.
--First, look up the name of the clustered rowstore index.
--Clustered rowstore indexes always use the DEFAULT name ‘ClusteredIndex_<GUID>’.
SELECT i.name
FROM sys.indexes i
JOIN sys.tables t
ON ( i.type_desc = 'CLUSTERED' ) WHERE t.name = 'MyDimTable';

--Drop the clustered rowstore index.
DROP INDEX ClusteredIndex_d473567f7ea04d7aafcac5364c241e09 ON MyDimTable;

  4. Convert the rowstore table to a columnstore table with a clustered columnstore index.

    Copy image Copy Code
    --Option 1: Convert to columnstore and name the new clustered columnstore index MyCCI.
CREATE CLUSTERED COLUMNSTORE INDEX MyCCI ON MyDimTable;

--Option 2: Convert to columnstore and use the rowstore clustered
--index name for the columnstore clustered index name.
--First, look up the name of the clustered rowstore index.
SELECT i.name
FROM sys.indexes i
JOIN sys.tables t
ON ( i.type_desc = 'CLUSTERED' )
WHERE t.name = 'MyDimTable';

--Second, create the clustered columnstore index and
--Replace ClusteredIndex_d473567f7ea04d7aafcac5364c241e09
--with the name of your clustered index.
CREATE CLUSTERED COLUMNSTORE INDEX
ClusteredIndex_d473567f7ea04d7aafcac5364c241e09
 ON MyDimTable
WITH DROP_EXISTING = ON;

C. Convert a large fact table from rowstore to columnstore

For best performance, use CREATE TABLE AS SELECT instead of CREATE COLUMNSTORE INDEX to convert fact and large dimension tables from rowstore to columnstore.

The following example creates a new table named myFactTable, using the column definitions and data from the source table dimCustomer.

Copy image Copy Code
USE SQLServerPDW2012;
CREATE TABLE myTable
WITH (
    CLUSTERED COLUMNSTORE INDEX,
    DISTRIBUTION = HASH (id),
)
AS SELECT *
FROM dimCustomer;

D. Create several small clustered columnstore indexes on some of the columns.

The best way to create a clustered columnstore index on only some of the table columns is to create the original table with a clustered columnstore index, and then use CREATE TABLE AS SELECT (CTAS) with the CLUSTERED COLUMNSTORE INDEX option to create an additional table with a clustered columnstore index.

For example, if you have a large table with many columns and plan to use CTAS to create smaller columnstore tables, you will achieve the best performance by first creating the larger table as a columnstore table.

It is faster to create additional clustered columnstore indexes from a columnstore table rather than from a rowstore table. When the table is already stored on the distributions as a columnstore, the data is already stored and compressed as columns. It is simple to copy the already compressed columns to a new table.

Conversely, if the original table is a rowstore, in order to create a new columnstore table, SQL Server PDW will convert the data to columnstore format for each new table, which takes more time.

Note that creating additional clustered columnstore indexes is only an option if you have additional storage available. This is because each additional clustered columnstore index is on a new columnstore table.

The following example creates a new table with a clustered columnstore index on only two columns. The full table is already a columnstore table with a clustered columnstore index.

Copy image Copy Code
--Recommended Method
--Creates a new fact table with fewer columns than the original fact table.
--Recommended method because ColumnStoreFactTable already has a clustered columnstore index.
CREATE TABLE ColumnStoreFactTable2
AS SELECT id, lastName
FROM ColumnStoreFactTable
WITH (
    DISTRIBUTION = HASH (id),
    CLUSTERED COLUMNSTORE INDEX
);

This next example takes more time to complete because the original table is a rowstore table.

Copy image Copy Code
CREATE TABLE RowstoreTable (
    id int NOT NULL,
    lastName varchar(20),
    zipCode varchar(6)
)
WITH ( DISTRIBUTION = HASH (id) );

CREATE TABLE ColumnstoreFactTable2
AS SELECT id, lastName
FROM RowstoreTable
WITH (
    DISTRIBUTION = HASH (id),
    CLUSTERED COLUMNSTORE INDEX
);

E. Rebuild a specific partition of the columnstore.

To rebuild a partition of a large clustered columnstore index, use ALTER INDEX REBUILD. You can also use ALTER INDEX with the REBUILD option to rebuild all partitions, or to rebuild a non-partitioned clustered columnstore index.

Copy image Copy Code
ALTER INDEX cci_fact3
ON fact3
REBUILD PARTITION = 12;

F. Rebuild the entire clustered columnstore index

There are two ways to rebuild the full clustered columnstore index. You can use CREATE CLUSTERED COLUMNSTORE INDEX, or ALTER INDEX and the REBUILD option. Both methods achieve the same results.

Copy image Copy Code
--Determine the Clustered Columnstore Index name of MyDimTable.
SELECT i.object_id, i.name, t.object_id, t.name
FROM sys.indexes i
JOIN sys.tables t
ON (i.type_desc = 'CLUSTERED COLUMNSTORE')
WHERE t.name = 'RowstoreDimTable';

--Rebuild the entire index by using CREATE CLUSTERED INDEX.
CREATE CLUSTERED COLUMNSTORE INDEX my_CCI
ON MyFactTable
WITH ( DROP_EXISTING = ON );

--Rebuild the entire index by using ALTER INDEX and the REBUILD option.
ALTER INDEX my_CCI
ON MyFactTable
REBUILD PARTITION = ALL
WITH ( DROP_EXISTING = ON );

G. Convert a columnstore table to a rowstore table with a clustered index

To convert a columnstore table to a rowstore table with a clustered index, use the CREATE INDEX statement with the DROP_EXISTING option.

Copy image Copy Code
CREATE CLUSTERED INDEX ci_MyTable
ON MyFactTable
WITH ( DROP EXISTING = ON );

H. Convert a columnstore table to a rowstore heap

To convert a columnstore table to a rowstore heap, simply drop the clustered columnstore index.

Copy image Copy Code
DROP INDEX MyCCI
ON MyFactTable;

[ TOP ]


Query Process (SQL Server PDW)

[image]

Query Process (SQL Server PDW)
Send Feedback

This topic describes how SQL Server PDW processes queries in parallel across the Compute nodes.

Understanding the Query Process

User-submitted SQL queries are processed by the Control node. The Control node engine parses the query and creates a query plan that defines the sequence of operations it will use to run the query on the appliance. The Control node distributed query plan operations run serially. When a query plan operation uses multiple parallel operations, the SQL Server PDW engine waits for all parallel operations to complete before starting the next distributed query plan operation.

Parallel Query Processing

And this is the query execution sequence:

SQL Server PDW Query Execution Sequence

For more information about queries, see the Query (SQL Server PDW) section of the documentation.


Minimum and Maximum Values (SQL Server PDW)

[image]

Minimum and Maximum Values (SQL Server PDW)
See Also Send Feedback

The following tables show boundaries and constraints on various SQL Server PDW objects and commands.

Contents

  • Minimum SQL Values

  • Maximum SQL Values

  • Maximum System View Values

  • Maximum Tools and Processes Values

Minimum SQL Values

The following table specifies the minimum numbers and sizes of various objects defined in SQL Server PDW databases or referenced in SQL statements.

Object or Command

Boundary

Minimum Constraint

Database

Compatibility level

120

All SQL Server PDW Appliance Update 3 user databases have database compatibility level 120, which is the compatibility level for SQL Server 2014. All databases from previous versions of PDW will have database compatibility level 120 after they have been upgraded or restored to SQL Server PDW Appliance Update 3 or a later appliance update.

Database

Replicated size.

0.0625 GB

The replicated size is specified as the amount allocated per Compute node. It is not the total amount allocated per appliance.

Database

Distributed size.

0.0625 GB per Compute node times the number of Compute nodes in the appliance.

0.0625 GB per Compute node = (2 drives per distribution) * (1 file per drive) * (4 MB per file) * ( 1 GB / 1024 MB) * (8 distributions per Compute node).

Database

Log size.

0.0625 GB per Compute node times the number of Compute nodes in the appliance.

Copy image Copy Code
--Create a database by using the minimum sizes
--for an appliance with 2 Compute nodes.

CREATE DATABASE TwoNodeMinSize
WITH (
    AUTOGROW=ON,
    REPLICATED_SIZE = 0.0625 GB,
    DISTRIBUTED_SIZE = 0.125 GB,
    LOG_SIZE = 0.125 GB
);

Index

Rows per CLOSED or COMPRESSED rowgroup in a clustered columnstore index.

102,400

This is the minimum number of rows that can be added to the clustered columnstore index at the same time.

[TOP]

Maximum SQL Values

The following table specifies the maximum numbers and sizes of various objects defined in SQL Server PDW databases or referenced in SQL statements.

Object or Command

Boundary

Maximum Constraint

Query

Concurrent queries on user tables.

32

Query

Queued queries on user tables.

1000

Query

Concurrent queries on system views.

100

Query

Queued queries on system views.

1000

Batch

Maximum size.

65,536 * 4096 bytes.

Query

Maximum parameters.

2098

Session

Open sessions.

1024

Session

Maximum memory for prepared statements.

20 MB

Database

Databases per SQL Server PDW instance.

32,767

Table

Tables per database.

2 billion

Table

Columns per table.

1024 columns

Table

Bytes per column

8000 bytes

Table

Bytes per row, SQL Server defined size

8060 bytes

This is a SQL Server requirement that the fixed size of each row must fit on one memory page. This limit is enforced in SQL Server PDW because SQL Server must be able to store each row. Bytes per row is computed the same as SQL Server with page compression ON.

SQL Server PDW will give a compile error if the size of the row exceeds the SQL Server maximum definition size of 8060 bytes.

  • To estimate the SQL Server row size, see the row size calculations in Estimate the Size of a Clustered Index on MSDN.

  • For a list of PDW data type sizes, see CREATE TABLE (SQL Server PDW).

Table

Bytes per row, DMS buffer defined size

32,768

DMS stores rows in a different format than SQL Server. To improve performance, DMS pads all variable length data to the maximum SQL Server defined size when it loads a row into the DMS buffer. For example, the value 'hello' for an nvarchar(2000) NOT NULL will actually use 4002 bytes in the DMS buffer since nvarchar NOT NULL types require 2 bytes per character plus 2 bytes for the NULL terminator.

For more information about the DMS buffer format, see A. Compute the DMS buffer row size in the Examples section.

Data movement operations will fail when a row’s defined DMS buffer size exceeds the DMS buffer size of 32,768. Loading data with dwloader and performing distribution incompatible joins are some examples of operations that require DMS to move data between nodes; these operations will fail when DMS tries to move a row that has a DMS buffer size of more than 32,768 bytes.

Both adding rows with the INSERT statement and loading data with Integration Services use a different code path than loading rows with dwloader. When using either of these to load, DMS moves the rows, but does not convert the row into the DMS buffer format. Therefore, it is possible to successfully load a row into PDW and then get a failure when DMS tries to move the row.

For example, you might be able to use INSERT or Integration Services to load a row that has a SQL Server defined row-size of 40,000 bytes, provided the actual content of the row does not exceed 32,768 bytes, and then get an error when DMS tries to move the row for a distribution-incompatible join.

For a detailed example, see B. Exceeding the DMS buffer size in the Examples section.

Table

Partitions per table.

15,000.

For high performance, we recommend minimizing the number of partitions you need while still supporting your business requirements. As the number of partitions grows, the overhead for DDL and DML operations grows and causes slower performance.

Table

Characters per partition boundary value.

4000

Index

Non-clustered indexes per table.

999

Applies to rowstore tables only.

Index

Clustered indexes per table.

1

Applies to both rowstore and columnstore tables.

Index

Rows per rowgroup in a clustered columnstore index.

1,048,576

This is the maximum number of rows that can be added to the clustered columnstore index at the same time. The number of rows per rowgroup has a minimum and a maximum.

Index

Concurrent builds of clustered columnstore indexes.

32

Applies when the clustered columnstore indexes are all being built on different tables. Only 1 clustered columnstore index build is allowed per table. Additional requests will wait in a queue.

Index

Index key size.

900 bytes.

Applies to rowstore indexes only.

Indexes on varchar columns that exceed 900 bytes can be created if the existing data in the columns does not exceed 900 bytes when the index is created. However, later INSERT or UPDATE actions on the columns that cause the total size to exceed 900 bytes will fail.

Index

Key columns per index.

16

Applies to rowstore indexes only. Clustered columnstore indexes include all columns.

Statistics

Size of the combined column values.

900 bytes.

Statistics

Columns per statistics object.

32

Statistics

Statistics created on columns per table.

30,000

Stored Procedures

Maximum levels of nesting.

8

View

Columns per view

1,024

Logins

Logins per SQL Server PDW instance.

500,000

SELECT results

Columns per row

4096

You can never have more than 4096 columns per row in the SELECT result. There is no guarantee that you can always have 4096. If the query plan requires a temporary table, the 1024 columns per table maximum might apply.

SELECT results

Bytes per row

>8060

The number of bytes per row in the SELECT result can be more than the 8060 byte maximum that is allowed for table rows. If the query plan for the SELECT statement requires a temporary table, the 8060 byte table maximum might apply.

SELECT results

Bytes per column

>8000

The number of bytes per column in the SELECT result can be more than the 8000 byte maximum that is allowed for table columns. If the query plan for the SELECT statement requires a temporary table, the 8000 byte table maximum might apply.

SELECT

Nested subqueries

32

You can never have more than 32 nested subqueries in a SELECT statement. There is no guarantee that you can always have 32. For example, a JOIN can introduce a subquery into the query plan. The number of subqueries can also be limited by available memory.

SELECT

Columns per JOIN

1024 columns

You can never have more than 1024 columns in the JOIN. There is no guarantee that you can always have 1024. If the JOIN plan requires a temporary table with more columns than the JOIN result, the 1024 limit applies to the temporary table.

SELECT

Bytes per GROUP BY columns.

8060

The columns in the GROUP BY clause can have a maximum of 8060 bytes.

SELECT

Bytes per ORDER BY columns

8060 bytes.

The columns in the ORDER BY clause can have a maximum of 8060 bytes.

INSERT

Bytes per column, fixed and variable width.

8000 bytes. Attempts to insert more bytes than defined for the column will result in an error.

INSERT

Bytes per row, variable width columns.

>8060 bytes. Data in excess of 8060 bytes is placed in overflow storage area.

UPDATE

Bytes per column, fixed and variable width.

8000 bytes. Attempts to update a column to a value requiring more bytes than defined for the column will result in an error.

Identifiers and constants per statement

Number of referenced identifiers and constants.

65,535

SQL Server limits the number of identifiers and constants that can be contained in a single expression of a query. This limit is 65,535. Exceeding this number results in SQL Server error 8632. For more information, see Error message when you run a query in SQL Server 2005: "Internal error: An expression services limit has been reached".

[TOP]

Maximum System View Values

System View | Maximum Rows
sys.dm_pdw_component_health_alerts | 10,000
sys.dm_pdw_dms_cores | 100
sys.dm_pdw_dms_workers | Total number of DMS workers for the most recent 1000 SQL requests.
sys.dm_pdw_dms_worker_pairs | 10,000
sys.dm_pdw_errors | 10,000
sys.dm_pdw_exec_requests | 10,000
sys.dm_pdw_exec_sessions | 10,000
sys.dm_pdw_request_steps | Total number of steps for the most recent 1000 SQL requests that are stored in sys.dm_pdw_exec_requests.
sys.dm_pdw_os_event_logs | 10,000
sys.dm_pdw_sql_requests | The most recent 1000 SQL requests that are stored in sys.dm_pdw_exec_requests.

[TOP]

Password Age, Tools, and Processes Values

Tool or Process

Boundary

Maximum Constraint

Client Tool

SQL Server Compatibility

SQL Server 2008 SP2

For SQL Server compatibility, SQL Server PDW reports the SQL Server product version by using a SQL Server 2008 Service Pack 2 version number. For example, the statement SELECT @@VERSION returns a version number incremented from 10.00.4000.00 which indicates SQL Server 2008 Service Pack 2 (SP2).

For a client tool that is maintaining compatibility with SQL Server, the version number implies the client should only send statements that are supported by SQL Server 2008 SP2. The version number also implies that if a client tool works with SQL Server 2008 SP2, it will work with SQL Server PDW, provided the client tool uses functionality that is available in PDW.For a list of differences between supported SQL Server and SQL Server PDW commands, see SQL Server to SQL Server PDW Migration Guide at http://upgradetopdw.com.

Some tools recognize the client is SQL Server PDW rather than an instance of SMP SQL Server. Such tools are not trying to maintain SQL Server compatibility, and so they issue statements according to what SQL Server PDW supports. For example, SQL Server Data Tools sends clustered columnstore index commands and other commands to PDW that are beyond the scope of SQL Server 2008 SP2.

Data Loads

Active loads per appliance.

10

The combined total of loads with dwloader and Integration Services is 10 active loads per appliance. When this maximum is exceeded, the loads will wait in the load queue.

For more information about how this maximum relates to the maximum number of destinations in an Integration Services PDW package, see the Limitations and Restrictions section in Load Data With Integration Services (SQL Server PDW).

Data Loads

Queued loads per appliance.

40

Loads are blocked from running if they cause the number of queued loads to exceed this maximum.

Data Loads

Files per load.

10,000

dwloader Command-Line Loader (SQL Server PDW) can load a maximum of 10,000 files from the same directory.

Database Backups

Active database backups per appliance.

1

Database Backups

Concurrent Compute node backups per database backup.

4

A user database backup requires a backup from each Compute node. SQL Server PDW allows 4 concurrent Compute node backups for each database backup.

Domain Administrator Password

Days per password.

60 days.

The domain administrator password for the appliance domain expires after 60 days.

Password age

Days per password

Minimum 1 day to maximum 42 days.

Minimum password length

Password characters

7 to 128 characters.

Caution note Caution

For more information on password policies, see CREATE LOGIN (SQL Server PDW).

[TOP]

Examples

A. Compute the DMS buffer row size

This example describes the DMS defined sizes of variable-length data and then calculates the DMS buffer size for a row.format.

In the DMS buffer, variable-length data size is the sum of the following:

  • Defined size for characters.

  • NULL types use 8 bytes for the NULL indicator.

  • ASCII types use 1 character for the NULL terminator.

  • Unicode types use 2 characters for the NULL terminator.

Data Type | DMS buffer size
char(1000) NULL | 1009 bytes = 1000 +8 + 1
char(1000) NOT NULL | 1001 bytes = 1000 + 1
nchar(1000 NULL | 2010 bytes = 2000 + 8 + 2
varchar(1000) NULL | 1009 bytes = 1000 + 8 + 1
varchar(1000) NOT NULL | 1009 bytes = 1000 + 1
nvarchar(1000) NULL | 2010 bytes = 2000 + 8 + 2
nvarchar(1000) NOT NULL | 2010 bytes = 2000 + 2

In the DMS buffer, fixed-width columns use the SQL Server native size. If the type is nullable, DMS requires an extra 8 bytes. For the SQL Server size, see the max_length field in sys.types (SQL Server PDW).

For example:

Fixed-Width Data Type | DMS buffer size
int NULL | 12 bytes = 4 + 8
int NOT NULL | 4 bytes = 4 + 0

Putting it altogether, the DMS buffer defined size for the following row is 31,134 bytes, which will fit in the DMS buffer.

Column | Data Type | Column Size
col1 | datetime2 (7) NULL | 20 bytes = 8 + 8
col2 | float (4) NULL | 12 bytes = 4 + 8
col3 | nchar (6) NULL | 22 bytes = 12 + 8 + 2
col4 | char (7000) NULL | 7009 bytes = 7000 + 8 + 1.
col5 | nvarchar (4000) | 8002 bytes = 8000 + 2.
col6 | varchar (8000) NULL | 8009 bytes = 8000 + 8 + 1
col7 | varbinary (8000) | 8000 bytes
col8 | binary (60) | 60 bytes

B. Exceeding the DMS buffer size

This example shows how you could successfully insert a row into PDW, but then incur a DMS overflow error when DMS needs to move the row for a distribution incompatible join. The lesson to be learned is to create smaller rows that will fit into the DMS buffer.

In the following example, we create table T1. The maximum possible size of the row when all the nvarchar variables have 4000 Unicode characters is more than 40,000 bytes, which exceeds the maximum DMS buffer size.

Since the actual defined size of an nvarchar uses 26 bytes, the row definition is less than 8060 bytes and can fit on a SQL Server page. Therefore the CREATE TABLE statement succeeds, even though DMS will fail when it tries to load this row into the DMS buffer.

Copy image Copy Code
CREATE TABLE T1
  (
    c0 int NOT NULL,
    CustomerKey int NOT NULL,
    c1 nvarchar(4000),
    c2 nvarchar(4000),
    c3 nvarchar(4000),
    c4 nvarchar(4000),
    c5 nvarchar(4000)
  )
WITH ( DISTRIBUTION = HASH (c0) )
;

This next step shows that we can successfully use INSERT to insert data into the table. This statement loads the data directly into SQL Server without using DMS, and so it does not incur a DMS buffer overflow failure. Integration Services will also load this row successfully.

Copy image Copy Code
--The INSERT operation succeeds because the row is inserted directly into SQL Server without requiring DMS to buffer the row.
INSERT INTO T1
VALUES (
    25,
    429817,
    N'Each row must fit into the DMS buffer size of 32,768 bytes.',
    N'Each row must fit into the DMS buffer size of 32,768 bytes.',
    N'Each row must fit into the DMS buffer size of 32,768 bytes.',
    N'Each row must fit into the DMS buffer size of 32,768 bytes.',
    N'Each row must fit into the DMS buffer size of 32,768 bytes.'
  )

To prepare for demonstrating data movement, this example creates a second table with CustomerKey for the distribution column.

Copy image Copy Code
--This second table is distributed on CustomerKey.
CREATE TABLE T2
  (
    c0 int NOT NULL,
    CustomerKey int NOT NULL,
    c1 nvarchar(4000),
    c2 nvarchar(4000),
    c3 nvarchar(4000),
    c4 nvarchar(4000),
    c5 nvarchar(4000)
  )
WITH ( DISTRIBUTION = HASH (CustomerKey) )
;

INSERT INTO T2
VALUES (
    32,
    429817,
    N'Each row must fit into the DMS buffer size of 32,768 bytes.',
    N'Each row must fit into the DMS buffer size of 32,768 bytes.',
    N'Each row must fit into the DMS buffer size of 32,768 bytes.',
    N'Each row must fit into the DMS buffer size of 32,768 bytes.',
    N'Each row must fit into the DMS buffer size of 32,768 bytes.'
  )

Since both tables are not distributed on CustomerKey, a join between T1 and T2 on CustomerKey is distribution incompatible. DMS will need to load at least one row and copy it to a different distribution.

Copy image Copy Code
SELECT * FROM T1 JOIN T2 ON T1.CustomerKey = T2.CustomerKey;

As expected, DMS fails to perform the join because the row, when all the nvarchar columns are padded, is larger than the DMS buffer size of 32,768 bytes. The following error message occurs.

Copy image Copy Code
Msg 110802, Level 16, State 1, Line 126
An internal DMS error occurred that caused this operation to fail. Details: Exception: Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DmsSqlNativeException, Message: SqlNativeBufferReader.ReadBuffer, error in OdbcReadBuffer: SqlState: , NativeError: 0, 'COdbcReadConnection::ReadBuffer: not enough buffer space for one row | Error calling: pReadConn->ReadBuffer(pBuffer, bufferOffset, bufferLength, pBytesRead, pRowsRead) | state: FFFF, number: 81, active connections: 8', Connection String: Driver={SQL Server Native Client 11.0};APP=DmsNativeReader:P13521-CMP02\sqldwdms (4556) - ODBC;Trusted_Connection=yes;AutoTranslate=no;Server=P13521-SQLCMP02,1500

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)


Plan for Backup and Loading Hardware (SQL Server PDW)

[image]

Plan for Backup and Loading Hardware (SQL Server PDW)
See Also Send Feedback

To deploy your end-to-end SQL Server PDW data warehousing solution, you need to create plans for backup and loading. Use this guidance to acquire and configure backup and loading servers that will meet your business requirements.

Acquire and Configure Backup Servers

SQL Server PDW Backup Process

To backup a SQL Server PDW database, you need one or more backup servers. You can use your own existing hardware or purchase new hardware. For more information, see Acquire and Configure a Backup Server (SQL Server PDW). These instructions include a Backup Server Capacity Planning Worksheet (SQL Server PDW) to help you plan the right solution for backup.

Acquire and Configure Loading Servers

To load data, you need one or more loading servers. You can use your own existing ETL or other servers, or you can purchase new servers. For more information, see Acquire and Configure a Loading Server (SQL Server PDW). These instructions include a Loading Server Capacity Planning Worksheet (SQL Server PDW) to help you plan the right solution for loading.

See Also

Concepts

Backup and Restore (SQL Server PDW)
Load (SQL Server PDW)


Acquire and Configure a Loading Server (SQL Server PDW)

[image]

Acquire and Configure a Loading Server (SQL Server PDW)
See Also Send Feedback

This topic describes how to acquire and configure a Loading Server as a non-appliance Windows system for submitting data loads to SQL Server PDW.

Contents

  • Basics

  • Step 1: Determine Capacity Requirements

  • Step 2: Acquire the Server

  • Step 3: Connect the Server to the InfiniBand Networks

  • Step 4: Install the Loading Tools

  • Step 5: Start Loading

  • Security Notices

Basics

The loading server:

  • Does not have to be a single server. You can load concurrently with multiple loading servers.

  • Is provided and managed by your own IT team. You might already have a server or servers that can be used for loading data into SQL Server PDW.

  • Is located in your own non-appliance rack, and cannot be placed within the Analytics Platform System appliance.

  • Is connected to the appliance through the Appliance InfiniBand network, or over Ethernet. For performance, we recommend using InfiniBand.

  • Is in your own customer domain, not the appliance domain. There is no trust relationship between your customer domain and the appliance domain.

[TOP]

Step 1: Determine Capacity Requirements

The loading system can be designed as one or more Loading servers that perform concurrent loads. Each Loading server does not have to be dedicated only to loading, as long as it will handle the performance and storage requirements of your workload.

The system requirements for a Loading server depend almost completely on your own workload. Use the Loading Server Capacity Planning Worksheet (SQL Server PDW) to help determine your capacity requirements.

Step 2: Acquire the Server

Now that you better understand your capacity requirements, you can plan the servers and networking components that you will need to purchase or provision. Incorporate the following list of requirements into your purchasing plan, and then purchase your server or provision an existing server.

Software Requirements

Supported Operating Systems:

  • Windows Server 2012 or Windows Server 2012 R2. These operating systems require the FDR network adapter.

  • Windows Server 2008 R2. This OS requires the DDR network adapter.

The server must use the EN-US locale in order to use the dwloader Command-Line Loading tool. dwloader does not support other locales.

Networking Requirements for Windows Server 2012 and Windows Server 2012 R2

Although not required for Loading, InfiniBand is the recommended connection type for Loading servers. For best performance, use Windows Server 2012 or Windows Server 2012 R2, and the FDR InfiniBand network adapter to connect the Loading server to the appliance InfiniBand network.

To prepare for a Windows Server 2012 or Windows Server 2012 R2 InfiniBand connection:

  1. Plan to rack the server close enough to the appliance so that you can connect it to the appliance InfiniBand switches. For more information from Mellanox Technologies about InfiniBand, see the whitepaper, Introduction to InfiniBand.

  2. Purchase a Mellanox ConnectX-3 FDR InfiniBand single or dual port network adapter. We recommend purchasing the network adapter with two ports for fault tolerance during data transmission. A two port network adapter is required for high availability.

  3. Purchase 2 FDR InfiniBand cables for a dual port card, or 1 FDR InfiniBand cable for a single port card. The FDR InfiniBand cables will connect the loading server to the appliance InfiniBand network. The cable length depends on the distance between the loading server and the appliance InfiniBand switches, according to your environment.

Step 3: Connect the Server to the InfiniBand Networks

Use these steps to connect the loading server to the InfiniBand network. If the server is not using the InfiniBand network, skip this step.

  1. Rack the server close enough to the appliance so that you can connect it to the appliance InfiniBand network.

  2. Install the InfiniBand Mellanox ConnectX-3 FDR InfiniBand network adapter into the loading server.

  3. Use the FDR cables to connect the InfiniBand network adapter to one of the two InfiniBand switches in the first appliance rack.

  4. Install and configure the appropriate Windows driver for the InfiniBand network adapter.

    • InfiniBand drivers for Windows are developed by the OpenFabrics Alliance, an industry consortium of InfiniBand vendors. The correct driver may have been distributed with your InfiniBand network adapter. If not, the driver can be downloaded from www.openfabrics.org.

  5. Configure the InfiniBand and DNS settings for the network adapters. For configuration instructions, see Configure InfiniBand Network Adapters (SQL Server PDW).

[TOP]

Step 4: Install the loading tools

To install dwloader, see Install dwloader Command-Line Loader (SQL Server PDW).

If you plan to use Integration Services for loading, you will need to install Integration Services and the Integration Services destination adapters. For instructions, see Install Integration Services Destination Adapters (SQL Server PDW).

[TOP]

Step 5: Start Loading

You are now ready to start loading data. For more information, see:

  1. dwloader Command-Line Loader (SQL Server PDW)

  2. Load (SQL Server PDW)

Performance

For best loading performance on Windows Server 2012 and beyond, turn on Instant File Initialization so that when data is overwritten, the operating system will not overwrite existing data with zeros. If this is a security risk because prior data still exists on the disks, then be sure to turn Instant File Initialization off.

Security Notices

Since the data to load is not stored on the appliance, your IT team is responsible for managing all aspects of the security for your data to load. For example, this includes managing the security of the data to load, the security of the server used to store loads, and the security of the networking infrastructure that connects the loading server to the SQL Server PDW appliance.

Important note Important

It is especially important to secure each loading server that will use dwloader Command-line Loading tool. When dwloader loads data, it first authenticates with the Control node, and then after successful authentication it moves data from the loading server directly to the Compute nodes over data channels. Certificate validation does not occur during the hand-shake between each loading server and each Compute node. This leaves the Compute nodes exposed to potential man-in-the-middle attacks on each data channel while loading. These attacks could result in tampered data and/or information disclosure.

To reduce security risks with your data, we advise the following:

  • Designate one Windows account that is used solely for the purpose of loading data into SQL Server PDW. Allow this account to have permissions to the load location and nowhere else.

  • Designate one SQL Server PDW user that has permissions to load data. Depending on your security requirements, you could have one specific user per database.

  • Operations on the loading server can accept an UNC path from which to pull data from outside the trusted internal network. And an attacker on the network or with ability to influence name resolution can intercept or modify data sent to the SQL Server PDW. This presents a tampering and information disclosure risk. Tampering should be mitigated by requiring signing on the connection. To help mitigate this risk, set the following group policy option in Security Settings\Local Policies\Security Options on the loading server: Microsoft network client: Digitally sign communications (always): Enabled

  • Turn off Instant File Initialization on Windows Server 2012 and beyond. This is a tradeoff between performance and security, as noted in the Performance section. You need to decide what is best according to your security requirements.

[TOP]

See Also

Concepts

Backup and Restore (SQL Server PDW)


Loading Server Capacity Planning Worksheet (SQL Server PDW)

[image]

Loading Server Capacity Planning Worksheet (SQL Server PDW)
See Also Send Feedback

This capacity planning worksheet helps you to determine the requirements for a loading server for loading data into SQL Server PDW. Use this to create your plan for purchasing or provisioning existing loading servers.

Capacity Planning Worksheet for Loading Servers

Worksheet Notes:

  1. This worksheet applies to servers that will be loading data with the dwloader Command-Line Loading Tool.

  2. For loading data with Integration Services or a third party loading tool, the requirements can vary depending upon differences in the loading process.

  3. Most of the requirements apply to loading either compressed or uncompressed data files; any differences in requirements are noted in bold.

icon used in topic Capacity Planning Worksheet

Print this worksheet and fill it in with your own requirements.

Component

Requirement

Fill in this column with your own requirements

Recommendations

Storage

Maximum bytes you plan to store on the Loading server at any given period of time.

pencil icon for topic

To determine storage requirements, figure out how much data you plan to store on the Loading server at any given period of time. Capacity requirements are for load files only; the operating system and the load files should be on different disk arrays.

For example:

  • If you plan to load 100GB of data from disk 3 times each day, but not delete the data files until the end of the week, then you would need a minimum 2.1TB of storage to store the data files.

  • We recommend being conservative and getting about 30% more to account for variances and growth. So for this example 2.73TB of storage space would be better.

Load Rate

Maximum bytes per hour of data to be loaded into SQL Server PDW.

pencil icon for topic

This is an estimate. When computing this requirement, assume that the files are already on the Loading server, and that other loading conditions are as good as possible.

For example:

  • No need to factor in data compressibility since dwloader always sends uncompressed data to the SQL Server PDW.

  • No need to factor in data type conversions and the size of the destination table.

Network

Network connection type.

pencil icon for topic

Determine the best network connection type that can meet your load rate requirements.

For example:

  • InfiniBand or 10Gbit Ethernet will provide the optimal loading rates.

  • 1Gbit Ethernet will limit load rates to 360 GB per hour or less.

I/O

Bytes per hour for reads and writes.

pencil icon for topic

To load data, dwloader must read all of the data from disk before sending it to SQL Server PDW.

Each loading server cannot load data faster than the appliance can receive data from all loading sources. To save money, plan the I/O read capacity for loading so that it does not exceed the load capacity of the appliance.

For example:

  • SQL Server PDW can receive and load data into a 1-rack appliance at a maximum rate of 1.8 TB per hour.

  • For an appliance with 2 or more racks, the maximum load rate is 3.6 TB per hour.

If you plan to load from multiple loading servers at the same time, the I/O requirements for each loading server will be less than when one server is doing all the loading.

For example:

  • One loading server can load a maximum of 1.8 TB per hour for a 1-rack appliance.

  • Two loading servers could each concurrently load 900 GB per hour into a 1-rack appliance.

  • Higher levels of concurrency can reduce efficiency and maximum throughput.

For I/O capacity, take into account all of the I/O happening on the loading server. If the Loading server has other I/O traffic in addition to data loads, such as receiving data files from an ETL server, the I/O requirements will increase.

For compressed data, the I/O requirements depend on the data compression rate. dwloader reads the compressed data and then uncompresses it before sending it to SQL Server PDW. The higher the compression ratio, the less data the Loading server will need to read from disk.

For example:

  • If the required load rate is 1.8 TB per hour, and the data is stored on the Loading server with 2:1 compression, then the Loading server needs only to read 900 GB per hour from disk instead of 1.8 TB.

  • A 3:1 compression ratio means the Loading server needs to read 600 GB per hour from disk.

CPU

Number of sockets.

 pencil icon for topic

For loading uncompressed data, dwloader is not a CPU-intensive application. As a minimum requirement, we recommend using a recently-manufactured 2-socket server.

For loading compressed data, you need enough CPU power to uncompress the data before sending it to SQL Server PDW. dwloader can run 10 active threads at once. If you plan to load 10 compressed files concurrently, we recommend the server has at least a 10-core CPU or two 6-core CPUs.

RAM

GB of memory that allows Windows to cache files during loads.

 pencil icon for topic

dwloader uses very little RAM on the Loading server. For performance, Windows uses memory to cache the loading files after they are read from disk.

To determine the RAM requirements, refer to your Windows Server installation and any 3rd party application requirements. We recommend a minimum of 32GB if you do not have requirements from other sources.

For compressed data, faster RAM is useful because it will speed up the decompression.

See Also

Reference

dwloader Command-Line Loader (SQL Server PDW)

Concepts

Acquire and Configure a Loading Server (SQL Server PDW)


Acquire and Configure a Backup Server (SQL Server PDW)

[image]

Acquire and Configure a Backup Server (SQL Server PDW)
See Also Send Feedback

This topic describes how to configure a non-appliance Windows system as a backup server for use with the backup and restore features in SQL Server PDW.

Contents

  • Basics

  • Step 1: Acquire a Backup Server

  • Step 2: Connect the Server to the InfiniBand Networks

  • Step 3: Configure the Backup File Share

  • Security Notices

Backup Server Basics

The backup server:

  • Is provided and managed by your own IT team.

  • Does not require any SQL Server PDW-specific software or tools. SQL Server PDW does not install any software onto the backup server.

  • Is located in your own non-appliance rack, and cannot be placed within the Analytics Platform System appliance.

  • Can be connected to the appliance InfiniBand network. Backups can be performed over InfiniBand or Ethernet; InfiniBand is recommended for performance reasons.

  • Is in your own customer domain, not the appliance domain. There is no trust relationship between your customer domain and the appliance domain.

  • Hosts a backup file share, which is a Windows file share that uses the Server Message Block (SMB) application-level network protocol. The backup file share is permissioned to give a Windows domain user (usually this is a dedicated backup user) the ability to perform backup and restore operations on the share. The username and password credentials of the Windows domain user are stored in SQL Server PDW so that SQL Server PDW can perform backup and restore operations on the backup file share.

[TOP]

Step 1: Determine Capacity Requirements

The system requirements for the Backup server depend almost completely on your own workload. Before purchasing or provisioning a backup server you need to figure out your capacity requirements. The Backup server does not have to be dedicated only to backups, as long as it will handle the performance and storage requirements of your workload. You can also have multiple backup servers in order to backup and restore each database to one of several servers.

Use the Backup Server Capacity Planning Worksheet (SQL Server PDW) to help determine your capacity requirements.

[TOP]

Step 2: Acquire the Server

Now that you better understand your capacity requirements, you can plan the servers and networking components that you will need to purchase or provision. Incorporate the following list of requirements into your purchasing plan, and then purchase your server or provision an existing server.

Software Requirements

Any file server that uses the Windows File Sharing (SMB) protocol.

We recommend Windows Server 2012 and beyond in order to:

  • Get the performance benefit of file pre-allocation over SMB.

  • Use Instant File Initialization (IFI) for backup operations. Your IT team manages this setting on the backup server. The SQL Server PDW Configuration Manager (dwconfig.exe) does not set or control IFI on your backup server. Previous versions of Windows do not have IFI, but can still be used as backup servers.

[TOP]

Networking Requirements

Although not required, InfiniBand is the recommended connection type for Backup servers. To prepare for connecting the loading server to the appliance InfiniBand network:

  1. Plan to rack the server close enough to the appliance so that you can connect it to the appliance InfiniBand switches. For more information from Mellanox Technologies about InfiniBand, see the whitepaper, Introduction to InfiniBand.

  2. Purchase a Mellanox ConnectX-3 FDR InfiniBand single or dual port network adapter. We recommend purchasing the network adapter with two ports for fault tolerance during data transmission. A two port network adapter is required for high availability.

  3. Purchase 2 FDR InfiniBand cables for a dual port card, or 1 FDR InfiniBand cable for a single port card. The FDR InfiniBand cables will connect the loading server to the appliance InfiniBand network. The cable length depends on the distance between the loading server and the appliance InfiniBand switches, according to your environment.

[TOP]

Step 3: Connect the Server to the InfiniBand Networks

Use these steps to connect the loading server to the InfiniBand network. If the server is not using the InfiniBand network, skip this step.

  1. Rack the server close enough to the appliance so that you can connect it to the appliance InfiniBand network.

  2. Install the InfiniBand Mellanox ConnectX-3 FDR InfiniBand network adapter into the loading server.

  3. Use the FDR cables to connect the InfiniBand network adapter to one of the two InfiniBand switches in the first appliance rack.

  4. Install and configure the appropriate Windows driver for the InfiniBand network adapter.

    • InfiniBand drivers for Windows are developed by the OpenFabrics Alliance, an industry consortium of InfiniBand vendors. The correct driver may have been distributed with your InfiniBand network adapter. If not, the driver can be downloaded from www.openfabrics.org.

  5. Configure the InfiniBand and DNS settings for the network adapters. For configuration instructions, see Configure InfiniBand Network Adapters (SQL Server PDW).

[TOP]

Step 4: Configure the backup file share

SQL Server PDW will access the backup server through a UNC file share. To set up the file share:

  1. Create a folder on the backup server for storing your backups.

  2. Create a file share, called a backup share, for the backup folder.

  3. Designate or create a Windows domain account in your customer domain that you want to use for the purposes of performing backups and restores. For security reasons, it is best to use a dedicated account as the backup user.

  4. Add permissions to the backup share so that only trusted accounts and a domain backup account can access, read, and write to the share location.

  5. Add the backup domain account credentials to SQL Server PDW.

    For example:

    Copy image Copy Code
    EXEC sp_pdw_add_network_credentials '10.192.147.63', 'seattle\david', '********';

    For more information, see these stored procedures:

    • sp_pdw_add_network_credentials (SQL Server PDW)

    • sp_pdw_remove_network_credentials (SQL Server PDW)

[TOP]

Step 5: Start Backing Up Your Data

You are now ready to start backing up data to your backup server.

To backup data, use a query client to connect to SQL Server PDW and then submit BACKUP DATABASE OR RESTORE DATABASE commands. Use the DISK= clause to specify the Backup server and backup location.

Important note Important

Remember to use the InfiniBand IP address of the Backup server. Otherwise, the data will be copied over Ethernet instead of InfiniBand.

For example:

Copy image Copy Code
BACKUP DATABASE Invoices TO DISK = '\\10.172.14.255\backups\yearly\Invoices2013Full';

RESTORE DATABASE Invoices2013Full
FROM DISK = '\\10.172.14.255\backups\yearly\Invoices2013Full'

For more information, see, and examples, see BACKUP DATABASE (SQL Server PDW)

  • BACKUP DATABASE (SQL Server PDW)

  • RESTORE DATABASE (SQL Server PDW)

[TOP]

Security Notices

The backup server is not joined to the private domain for the appliance. It is in your own network, and there is no trust relationship between your own domain and private appliance domain.

Since SQL Server PDW backups are not stored on the appliance, your IT team is responsible for managing all aspects of the backup security. For example, this includes managing the security of the backup data, the security of the server used to store backups, and the security of the networking infrastructure that connects the backup server to the SQL Server PDW appliance.

Manage Network Credentials

Network access to the backup directory is based on standard Windows file sharing security. Before performing a backup, you need to create or designate a Windows account that will be used for authenticating SQL Server PDW to the backup directory. This windows account must have permission to access, create, and write to the backup directory.

Important note Important

To reduce security risks with your data, we advise that you designate one Windows account solely for the purpose of performing backup and restore operations. Allow this account to have permissions to the backup location and nowhere else.

You need to store the user name and password in SQL Server PDW by running the sp_pdw_add_network_credentials (SQL Server PDW) stored procedure. SQL Server PDW uses Windows Credential Manager to store and encrypt user names and passwords on the Control node and Compute nodes. The credentials are not backed up with the BACKUP DATABASE command.

To remove network credentials from SQL Server PDW, see sp_pdw_remove_network_credentials (SQL Server PDW)

To list all of the network credentials stored in SQL Server PDW, use the sys.dm_pdw_network_credentials (SQL Server PDW) dynamic management view.

Secure Communications

Operations on the loading server can accept an UNC path from which to pull data from outside the trusted internal network. And an attacker on the network or with ability to influence name resolution can intercept or modify data sent to the SQL Server PDW. This presents a tampering and information disclosure risk. Tampering should be mitigated by requiring signing on the connection. To help mitigate this risk, set the following group policy option in Security Settings\Local Policies\Security Options on the loading server: Microsoft network client: Digitally sign communications (always): Enabled

[TOP]

See Also

Concepts

Backup and Restore (SQL Server PDW)


Backup Server Capacity Planning Worksheet (SQL Server PDW)

[image]

Backup Server Capacity Planning Worksheet (SQL Server PDW)
See Also Send Feedback

This capacity planning worksheet helps you to determine the requirements for a backup server for performing SQL Server PDW database backup and restore operations. Use this to create your plan for purchasing new or provisioning existing backup servers.

This worksheet is a supplement to the instructions in Acquire and Configure a Backup Server (SQL Server PDW).

Capacity Planning Worksheet for Backup Servers

Worksheet Notes:

  1. This worksheet applies to servers that will be performing backup and restore operations for SQL Server PDW databases.

  2. The only way to backup and restore SQL Server PDW databases is to use the BACKUP DATABASE and RESTORE DATABASE SQL commands. However, once the backup data is on your backup server, it exists as a set of Windows files. You can archive the backup files from your server to another storage location by using traditional Windows file-based backup methods.

icon used in topic Capacity Planning Worksheet

Print this worksheet and fill it in with your own requirements.

Component

Requirement

Fill in this column with your own requirements

Recommendations

Storage

Maximum bytes you plan to store on the backup server at any given period of time.

pencil icon for topic

To determine storage requirements, figure out how much data you plan to store on the backup server at any given period of time.

The backup data is stored on the backup server in a compressed format. Data compression rates depend on the characteristics of your data.

For example:

  • As a rough estimate, we recommend estimating a 7:1 compression ratio relative to the size of your uncompressed data. This assumes at least 80% of the data is stored in clustered columnstore indexes.

  • For example, if you have 700 GB of uncompressed data in a database and it is stored in clustered columnstore indexes, then you could estimate the database backup will need about 100 GB.

If you plan to have multiple copies of database backups on the backup server, you need to account for them.

For example:

  • If you plan to backup 10 databases that each contain 5 TB of uncompressed data, the databases have a combined size of 50 TB. If you plan backup these 10 databases everyday for 5 days in a row, the total uncompressed size is 250 TB. Factoring in a 7:1 compression ratio, you will need 250 / 7 = 35.7 TB of storage on your backup server.

  • We recommend being conservative and getting about 30% additional capacity to account for variances and grow. In this example, 46.6 TB would be better.

Network

Network connection type.

pencil icon for topic

Determine the best network connection type that can meet your load rate requirements.

For example:

  • InfiniBand or 10Gbit Ethernet will provide the optimal loading rates.

  • 1Gbit Ethernet will limit load rates to 360 GB per hour or less.

I/O

Bytes per hour for writes.

pencil icon for topic

For writing backups to disk, 4 TB per hour write speeds are optimal.

For example:

  • For drives that can write 50 MB/sec, you will want at least 24 drives, plus more for mirroring or parity.

For I/O capacity, take into account all of the I/O happening on the loading server. If the Loading server has other I/O traffic in addition to data loads, such as receiving data files from an ETL server, the I/O requirements will increase.

CPU

Number of sockets.

 pencil icon for topic

Receiving and storing backup files is not a CPU-intensive application. As a minimum requirement, we recommend using a recently-manufactured 2-socket server.

RAM

GB of memory that allows Windows to cache files during loads.

 pencil icon for topic

Receiving and storing backup files requires very little RAM on the Loading server.

To determine the RAM requirements, refer to your Windows Server installation and any 3rd party application requirements. We recommend a minimum of 32GB if you do not have requirements from other sources.

When you are finished determining your capacity requirements, return to the Acquire and Configure a Loading Server (SQL Server PDW) topic to plan your purchase.

See Also

Concepts

Plan for Backup and Loading Hardware (SQL Server PDW)


Configure InfiniBand Network Adapters (SQL Server PDW)

[image]

Configure InfiniBand Network Adapters (SQL Server PDW)
See Also Send Feedback

Describes how to configure the InfiniBand network adapters on a non-appliance client server to connect to the Control node on SQL Server PDW. Use these instructions for basic connectivity and for high availability, so that loading, backup, and other processes will automatically connect to the active InfiniBand network.

Contents

  • Description

  • Before You Begin

  • Step 1: Obtain the Appliance InfiniBand Network Settings

  • Step 2: Configure the InfiniBand Network Adapter on Your Client Server

Description

These instructions show you how to find and then set the correct InfiniBand IP addresses and subnet masks on your InfiniBand-connected server. They also explain how to set your server to use the SQL Server PDW DNS so that your connection will resolve to the active InfiniBand network.

For high availability, SQL Server PDW has two InfiniBand networks, one active and one passive. Each InfiniBand network has a different IP address for the Control node. If the active InfiniBand network goes down, the passive InfiniBand network becomes the active network. When this happens a script or process should automatically connect to the active InfiniBand network without changing the script parameters.

Specifically, in this topic you will:

  1. Find the InfiniBand IP addresses of the SQL Server PDW appliance DNS servers (appliance_domain-AD01 and appliance_domain-AD02). To do this you will log into the AD01 and AD02 servers and get the IP addresses for each InfiniBand network. The InfiniBand IP addresses on the AD node are the DNS IP addresses. Find the InfiniBand IP addresses of the Analytics Platform System DNS servers (appliance_domain-AD01 and appliance_domain-AD02). To do this you will log into both AD01 and AD02 servers and determine the IP addresses for each InfiniBand network. The InfiniBand IP addresses on the AD01 and AD02 servers are the DNS Server IP addresses.

  2. Configure each network adapter to use an available IP address on the Analytics Platform System InfiniBand networks.

    1. If you have two InfiniBand network adapters, you will configure one adapter with an available IP address in the first InfiniBand network which is called TeamIB1, and the other adapter with an available IP address in the second InfiniBand network which is called TeamIB2. Use the appliance_domain-AD01 TeamIB1 IP address as the preferred DNS server and appliance_domain-AD02 TeamIB1 IP address as the alternate DNS server for TeamIB1 network adapter. Use the appliance_domain-AD01 TeamIB2 IP address as the preferred DNS server and appliance_domain-AD02 TeamIB2 IP address as the alternate DNS server for TeamIB2 network adapter.

    2. If you have only one InfiniBand network adapter, you will configure the adapter with an available IP address from one of the InfiniBand networks. Then you will configure the preferred and the alternate DNS servers on this adapter using either appliance_domain-AD01 TeamIB1 and appliance_domain-AD02 TeamIB1 or using appliance_domain-AD01 TeamIB2 and appliance_domain-AD02 TeamIB2 whichever is on the same network as the configured adapter as the preferred and the alternate DNS servers respectively.

  3. Configure your InfiniBand network adapter to use Analytics Platform System DNS servers to resolve your connection to the active InfiniBand network.

    1. To configure this you will use the advanced TCP/IP settings to add the appliance domain DNS suffix to the beginning of the list of DNS suffixes on your client server. This only needs to be configured on one of the network adapters; the setting will apply to both adapters.

After configuring your InfiniBand network adapters, client processes can connect to the Control node on the InfiniBand network by using PDW_region-SQLCTL01 for the address of the server. Your server will append the Analytics Platform System DNS suffix, or you can enter the complete address which is PDW_region-SQLCTL01.appliance_domain.pdw.local.

For example, if your PDW region name is MyPDW and the appliance name is MyAPS, the dwloader server specification for loading data will either of the following:

  • dwloader –S MYPDW-SQLCTL01.MyAPS.pdw.local

  • dwloader –S MYPDW-SQLCTL01

[ TOP ]

Before You Begin

Requirements

You will need an Analytics Platform System appliance domain account to login to the AD01 node. For example, F12345\Administrator.

You will need a Windows account on the client server that has permission to configure the network adapters.

Prerequisites

These instructions assume the client server is already racked and cabled to the appliance InfiniBand network. For racking and cabling instructions, see Acquire and Configure a Loading Server (SQL Server PDW).

General Remarks

By using SQLCTL01, the Analytics Platform System DNS will connect your client server to the Control node by using the active InfiniBand network. This applies only to getting connected; if the InfiniBand network goes down during a load or backup, you will need to restart the process.

To meet your own business requirements, you can also join the client server to your own non-appliance workgroup or Windows domain.

[ TOP ]

Step 1: Obtain the Appliance InfiniBand Network Settings

To obtain the appliance InfiniBand network settings

  1. Login to the appliance AD01 node by using the appliance_domain\Administrator account.

  2. On the appliance AD01 node, open the Control Panel, select Network and Internet, select Network and Sharing Center, and then select Change Adapter Settings.

  3. In the Network Connections window, right-click on Team IB1 and select Properties.

    InfiniBand Connections on the Management node.

  4. From the Internet Protocol Version 4 (TCP/IPv4) Properties window, write down the values for the IP address and Subnet mask. The IP address of the appliance_domain-AD01 node is the IP address of the Analytics Platform System DNS server.

  5. Repeat steps 1-5 above for the TeamIB1 adapter on appliance_domain-AD02 server.

    PDW Management Node InfiniBand 1 Properties

  6. Click Cancel to close the window.

  7. Find an unused IP address on the TeamIB1 network, and write it down.

    1. To find an unused IP address, open a command window and try to ping IP addresses within the range of addresses for your appliance. In this example, the IP address of the TeamIB1 network is 172.16.14.30. Find an IP address that starts with 172.16.14 that is not used. For example, from the command-line enter "ping 172.16.14.254". If the ping request is unsuccessful, the IP address is available.

  8. Do the same thing for TeamIB2. In the Network Connections window, right-click on Team IB2 and select Properties.

  9. From the Internet Protocol Version 4 (TCP/IPv4) Properties window, write down the values for the IP address and Subnet mask for TeamIB2.

  10. Repeat steps 8-9 above for the TeamIB2 adapter on appliance_domain-AD02 server.

    Properties for TeamIB2

  11. Find an unused IP address on the TeamIB2 network, and write it down.

    1. To find an unused IP address, open a command window and try to ping IP addresses within the range of addresses for your appliance. In this example, the IP address of the TeamIB2 network is 172.16.18.30. Find an IP address that starts with 172.16.18 that is not used. For example, from the command-line enter "ping 172.16.18.254". If the ping request is unsuccessful, the IP address is available.

[ TOP ]

Step 2: Configure the InfiniBand Network Adapter Settings on Your Client Server

Notes:

While performing these instructions:

  • You will register your server with Analytics Platform System DNS servers.

  • To meet your own network needs, you can also join the client server to your own non-appliance workgroup or Windows domain.

  • The instructions step through configuring two network adapters on each server. If you only have one network adapter, pick one of the networks to configure on the network adapter, and then add the second DNS IP address as an alternate DNS server.

To configure the InfiniBand network adapter settings on your client server

  1. Login as a Windows administrator to your loading, backup, or other client server on the appliance InfiniBand network.

  2. Open the Control Panel, select Network and Sharing Center, and then select Change Adapter Settings.

To configure the first network adapter

  1. In the Network Connections window, right-click on one of the unidentified network slots for the Mellanox Adapter and select Properties.

    Select the InfiniBand networks

  2. In the Properties window

    1. On the General tab, set the IP address to the IP address that you verified as free in the ping test for TeamIB1. For the example values used in this topic, you would enter 172.16.14.254.

    2. Set the subnet mask to the subnet mask you wrote down for TeamIB1.

    3. Set the Preferred DNS server to the IP address of TeamIB1 that you wrote down earlier from the appliance_domain-AD01 node.

    4. Set the Alternate DNS server to the IP address of TeamIB1 that you wrote down earlier from the appliance_domain-AD02 node.

      InfiniBand 1 Network Adapter Properties

    5. Click OK to apply the changes.

To configure the second network adapter

  1. Skip this section if you only have one network adapter.

  2. In the Network Connections window, right-click on the second unidentified network slot for the Mellanox Adapter and select Properties.

    Select the InfiniBand networks

  3. In the Properties window

    1. On the General tab, set the IP address to the IP address that you verified as free in the ping test for TeamIB2. For the example values used in this topic, you would enter 172.16.18.254.

    2. Set the subnet mask to the subnet mask you wrote down for TeamIB2.

    3. Set the Preferred DNS server to the IP address of TeamIB2 that you wrote down earlier from the appliance_domain-AD01 node.

    4. Set the Alternate DNS server to the IP address of TeamIB2 that you wrote down earlier from the appliance_domain-AD02 node.

      Note Note

      If you only have one network adapter, configure the preferred and the alternate DNS servers using either the appliance AD01 TeamIB1 and appliance AD02 TeamIB1 as the preferred and the alternate DNS servers respectively, or use the appliance AD01 TeamIB2 and appliance AD02 TeamIB2 as the preferred and the alternate DNS servers depending on if the AD virtual machine has failed over.

      InfiniBand 1 Network Adapter Properties

    5. Click OK to apply the changes.

To configure the DNS suffix

  1. In the Network Connections window, right-click on one of the network slots for the Mellanox Adapter and select Properties.

  2. Click on the Advanced… button.

  3. In the Advanced TCP/IP Settings window, if the Append these DNS Suffixes (in order) option is not greyed out, check the box called Append these DNS suffixes (in order):, select the appliance domain suffix, and click Add…. The appliance domain suffix will be appliance_domain.local

  4. If the Append these DNS suffixes (in order): option is greyed out, you can add the Analytics Platform System domain to this server by modifying the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\DNSClient.

  5. For faster address resolution, we recommend moving the appliance suffix to the top of the list.

  6. Click OK.

  7. Now, you can connect to the appliance Infiniband network by using PDW_region-SQLCTL01.appliance_domain.local, or simply appliance_domain-SQLCTL01. The connection might be established faster if you connect with the full name and DNS suffix.

    Examples for an appliance named named MyAPS with a MyPDW PDW region:

    • MyPDW-SQLCTL01.MyAPS.local

    • MyPDW-SQLCTL01

[ TOP ]

See Also

Concepts

Acquire and Configure a Loading Server (SQL Server PDW)


Client Tools and Applications (SQL Server PDW)

[image]

Client Tools and Applications (SQL Server PDW)
Send Feedback

Install client tools and applications that connect to SQL Server PDW, and also download the product documentation. Use the client tools to create an end-to-end data warehousing solution, built on SQL Server PDW.

Contents

  • Download the Product Documentation

  • Install PDW-Based Client Tools

  • Install SQL-Based Client Tools

  • Locate Appliance Management Tools

  • Limitations and Restrictions

  • Accessibility Notice

Download the Product Documentation

This product documentation CHM file is available from the Microsoft Download Center.

Note Note

The content is downloaded as a CHM file which might be blocked from use on your system. If you open the documentation file, and it is empty, you need to unblock the content. To do this, right-click the CHM file, select the General Tab, and at the bottom click Unblock. Now you should be able to open the CHM file and see all of the content.

Install PDW-Based Client Tools

The Appliance Update 3 client tools, written specifically for PDW, are available as free downloads from the Microsoft Download Center.

Download the tools, and then see these topics for installation and configuration instructions.

  • Install dwloader Command-Line Loader (SQL Server PDW) – the command line loading tool.

  • Install Integration Services Destination Adapters (SQL Server PDW) – for loading data into PDW by using Integration Services.

  • Install the Informatica Connector (SQL Server PDW) – for using Informatica with PDW.

  • Install AdventureWorksPDW2012 (SQL Server PDW) – to see a sample PDW database and to run code samples in the product documentation.

Install SQL-Based Client Tools

You might already have these SQL Server tools. They also work with PDW. If you don’t have them, use these links to install them.

  • Install SQL Server database tooling for Visual Studio (SQL Server PDW) – the graphical interface for querying PDW.

  • Install SQL Server Native Client (SQL Server PDW) – prerequisite for running SQL Server tools that work with PDW.

  • Install sqlcmd Command-Line Client (SQL Server PDW) – command-line interface for SQL Server that also works with PDW.

Locate Appliance Management Tools

These tools are already installed with your APS appliance and are ready to use for monitoring and managing APS.

  • Launch the Configuration Manager (Analytics Platform System)

  • Monitor the Appliance by Using the Admin Console (Analytics Platform System)

  • HDInsight Region Developer Dashboard (Analytics Platform System)

Limitations and Restrictions

For SQL Server compatibility, SQL Server PDW reports the SQL Server product version by using a SQL Server 2008 Service Pack 2 version number. Therefore, tools that connect to PDW as an instance of SQL Server will use operations that are compatible with SQL Server 2008. For more information, see Client Tools entry in Minimum and Maximum Values (SQL Server PDW).

Accessibility Notice

The Admin Console and dwloader applications meet Microsoft's standard for interacting with the application with the mouse and keyboard both together and independently. These applications support high-contrast displays and support accessibility assistance applications such as Narrator. These applications have multiple methods of interaction with the user interface and that the presentation of information is unambiguous and in multiple formats to ensure that regardless of customer needs the experience is consistent and without limit or restrictions.


Install SQL Server database tooling for Visual Studio (SQL Server PDW)

[image]

Install SQL Server database tooling for Visual Studio (SQL Server PDW)
See Also Send Feedback

Visual Studio provides SQL Server database tooling (formerly called SQL Server Data Tools) in an integrated environment for database developers to carry out all their database design work for any SQL Server platform (both on- and off-premises), all from within Visual Studio. Use this, as our recommended GUI query client SQL Server PDW.

Contents

  • Download and Install SQL Server Data Tools for Visual Studio

  • Create a Connection

  • Run a query

  • Limitations and Restrictions

  • Examples and Tips

Download and Install SQL Server Database Tooling for Visual Studio

SQL Server database tooling, formerly called SQL Server Data Tools, is available in Visual Studio 2012 and 2013. AU2 support is included in the July 2014 update for both versions.

If you have Visual Studio 2013 Ultimate, Premium, or Professional, you will only need to update SQL Server database tooling component. If you have Visual Studio 2013 Express for Windows Desktop, you will also need to apply Visual Studio Update 3 in addition to the SQL Server database tooling update. We recommend that you install Visual Studio Update 3, regardless of the edition you have to ensure you have the latest bug fixes from Visual Studio.

To apply the updates to Visual Studio 2013:

  1. Open Visual Studio and go to the Tools > Extensions and Updates… menu.

  2. In the Extensions and Updates dialog, click on Updates. If you do not have either of the updates applied, you should see Microsoft SQL Server Update for database tooling and Visual Studio 2013 Update 3 in the list.

  3. Apply both of these updates.

  4. Restart Visual Studio after the updates are applied if it was not closed during the update.

If you have Visual Studio 2012, install the latest download of SQL Server tooling from SQL Server Data Tools for Visual Studio 2012.

Note that if you do not already have Visual Studio 2012, the SSDT installer will install everything it needs, including the Visual Studio Integrated Shell. If you already have Visual Studio 2012, you probably already have SQL Server Data Tools. It is installed with Visual Studio, unless you unselected it during setup.

Visual Studio 2010 is not supported for Analytics Platform System Appliance Update 2.

To install SQL Server tooling onto a computer that does not have internet access, see the instructions on setting up an Administrative Install Point on the Get Started with Microsoft SQL Server Data Tools page on MSDN.

TOP

Create a Connection

To connect to your SQL Server PDW appliance:

  1. Open Visual Studio.

    • If you have Visual Studio 2013 or Visual Studio 2012, open it from the Start menu.

    • If you installed SSDT and Visual Studio 2012 was not already installed, open SQL Server Data Tools 2012 from the Start menu.

  2. If the SQL Server Object Explorer window is not visible, open SQL Server Object Explorer. To do this, click View and select SQL Server Object Explorer.

  3. In the SQL Server Object Explorer window, right-click SQL Server, and select Add SQL Server ….

  4. In the Connect to Server window enter the following:

    • Server name: Enter the appliance Infiniband DNS name of the Control node, followed by a comma (,), followed by port 17001. For example, pdw1-CTRL,17001. To enter a friendly name, or to enter the server name if the Control node does not have a DNS name, see Tip C in Tips and Examples at the end of this topic.

    • Authentication: Choose Windows Authentication, or SQL Server Authentication.

    • Login: Enter your user name if using SQL Server Authentication.

    • Password: Enter your password if using SQL Server Authentication.

  5. Click Connect.

  6. If the connection is successful, your appliance connection will appear in the SQL Server list in SQL Server Object Explorer.

TOP

Run a Query

  1. Open the query editor. To do this, expand your SQL Server PDW appliance node in SQL Server Object Explorer, expand the Databases folder right-click on the Database you want to query, and select New Query.

  2. Enter the query in the query editor window.

  3. To run the query, click the Execute button.

    Running a query on SQL Server PDW

TOP

Limitations and Restrictions

Some SQL Server Data Tools features are not supported in this release of SQL Server PDW. For example, these features are not supported:

  • IntelliSense

  • Power buffer

  • Offline DB projects

  • Schema compare

  • Query debugging

The SQL Server Data Tools options apply to all of your SQL Server connections including SQL Server PDW. Options that are not supported with SQL Server PDW connections will be ignored. To view or set the SQL Server Data Tools options, in the main menu click Tools, Options…, and select SQL Server Tools.

You cannot use SQL main menu to open a new query window for SQL Server PDW queries. For example, if you click the SQL menu, Transact-SQL Editor, and New Query… , SQL Server Data Tools will open the Transact-SQL Editor which does not query SQL Server PDW databases. Instead, follow the instructions in Step 3: Run a Query to open the SQL editor for SQL Server PDW.

TOP

Tips and Examples

A. How to Open SQL Server Data Tools for Visual Studio 2012

SQL Server Data Tools might appear in multiple places among the Programs in the All Programs menu in Windows.

Caution note Caution

You cannot use the SQL Server 2012 SQL Server Data Tools to browse database objects and to run SQL queries. This version of SQL Server Data Tools that installs with SQL Server 2012 contains the tools for Business Intelligence projects, and does not contain the tools for browsing and querying databases.

To open SQL Server Data Tools for the purpose of running SQL queries on SQL Server 2012 Parallel Data Warehouse, select any of the following links in the Windows Start menu.

  • All Programs | SQL Server Data Tools 2012. You get this if you install SQL Server Data Tools for Visual Studio 2012, and Visual Studio 2012 is not already installed.

  • All Programs | Microsoft Visual Studio 2012 | Visual Studio 2012.

B. Change Keyboard Shortcuts

You may want to change some of the pre-defined keyboard shortcuts in SQL Server Data Tools. For example, pressing Ctrl+Shift+E in the Query Editor Window will run the query. Users who are accustomed to SQL Server Management Studio might prefer pressing F5 to run a query.

The following example shows how to change the keyboard shortcut for running a query in SQL Server Data Tools from CTRL+SHIFT+E to F5.

  1. From the main menu, click Tools, Options…, Environment, Keyboard.

  2. In the Show Commands Containing field:, type ExecuteQuery to search, and select SQL.TSqlEditorExecuteQuery from the list.

  3. For Use new shortcut in:, choose Microsoft SQL Server Data Tools, T-SQL Editor.

  4. In the Press shortcut keys: field, press F5, or any other key combination that you want to assign to this action.

  5. Click Assign, and then click OK.

C. Assign a friendly name to a connection

Although you cannot change the name of the connection to a friendly name from within SQL Server Data Tools, you can create a connection that has a friendly name by editing the Windows system hosts file before creating the connection.

This is useful when the connection name is the IP address of your server, and you want to change it to a friendly name. For example, if your server does not have an assigned name in DNS, SQL Server Data Tools will display the IP address of your server as the connection name.

To assign a friendly name to an IP address in the Windows system hosts file

  1. Open a command prompt as an administrator. To do this, click Start, All Programs, and Accessories. Under Accessories, right-click Command Prompt and select Run as Administrator.

  2. To edit the list of host names, enter notepad %SystemRoot%\System32\drivers\etc\hosts on the command window and press Enter.

  3. Add a new line to the hosts file that specifies the IP address of your appliance, followed by a friendly name for the connection. For example, add the line: xxx.xxx.xxx.xxx SalesPDW.

  4. Save the file and exit notepad.

To assign a friendly name to a new connection

  1. Create a new connection as described by Step 2: Create a connection.

  2. In the connection window, instead of entering xxx.xxx.xxx.xxx,17001 as the connection name, enter SalesPDW,17001.

TOP

D. View SQL Server Data Tools Version Information

To verify which version of SQL Server Data Tools is installed

  1. Open SQL Server Data Tools for Visual Studio 2012 or open Visual Studio 2013.

  2. From the main menu, click Help, and select About Microsoft Visual Studio.

  3. Find SQL Server Data Tools in the list and view the version information.

SQL Server Data Tools Version

TOP

See Also

Concepts

Client Tools and Applications (SQL Server PDW)

Other Resources

Client Tools (SQL Server PDW)


Install Integration Services Destination Adapters (SQL Server PDW)

[image]

Install Integration Services Destination Adapters (SQL Server PDW)
See Also Send Feedback

This topic describes how to install the Integration Services Destination Adapters for Parallel Data Warehouse.

Parent Topic: Client Tools and Applications (SQL Server PDW)

Contents

  • Before You Begin

  • Step 1: Obtain the Installation Files

  • Step 2: Install the Destination Adapters

  • Step 3: Verify the Installation

  • Step 4: Add SQL Server PDW as a Data Flow Destination (SQL Server 2008 R2 only)

Basics

The destination adapters enable you to load data into SQL Server PDW by using Integration Services. They need to be installed onto the computer that uses Integration Services to build and run loading packages.

Before You Begin

Supported Operating Systems

Windows 7, Windows Server 2008, Windows Server 2012, Windows Server 2012 R2.

Supported Appliances

To load data into Appliance Update 3, use the Appliance Update 3 destination adapters. There are 32-bit and 64-bit Appliance Update 3 destination adapter for SQL Server 2008 R2, SQL Server 2012, and SQL Server 2014.

Software Prerequisites

If not already installed, install Microsoft .NET Framework 3.5 Service Pack 1.

Integration Services for any of these versions of SQL Server.

  • SQL Server 2014

  • SQL Server 2012 Service Pack 1 plus Cumulative Update Package 2, or later.

  • SQL Server 2012 plus Cumulative Update Package 5, or later.

    Important note Important

    If the Cumulative Update Packages are not applied, SQL Server 2012 Integration Services will load files serially instead of in parallel.

  • SQL Server 2008 R2

Step 1: Obtain the Installation files

Obtain the Windows Installer files for the destination adapters from your appliance administrator, or see Client Tools and Applications (SQL Server PDW) for the location of the tools. You will need both the 32-bit and 64-bit installation files for your version of Integration Services.

General Remarks

We recommend installing both the 32-bit and the 64-bit destination adapters for the following reasons:

  • Integration Services packages are usually created by using SQL Server Business Intelligence Development Studio (BIDS) for SQL Server 2008 R2, or SQL Server Data Tools (SSDT) for SQL Server 2012, or SQL Server data tooling for Visual Studio 2013 These are all 32-bit programs. Therefore, to create integration services packages with BIDS, SSDT, or SQL Server data tooling, you need to install the 32-bit destination adaptor.

  • Integration Services packages are usually run with 64-bit binaries for performance reasons. Therefore, to run an Integration Services package efficiently, you need to install the 64-bit destination adaptor. For more information, see 64-Bit Considerations for Integration Services.

[TOP]

Step 3: Install the Integration Services Destination Adapters

Install the Destination Adapters onto the computers that will use Integration Services to build or run Integration Services packages that load data into SQL Server PDW.

To install the Destination Adapters

  • For SQL Server 2008 R2 Integration Services, run the following Windows Installer files:

    • SSISSQLPDWDest-x86.msi

    • SSISSQLPDWDest-x64.msi

  • For SQL Server 2012 Integration Services, run the following Windows Installer files:

    • SSISSQL2012PDWDest-x86.msi

    • SSISSQL2012PDWDest-x64.msi

  • For SQL Server 2014 Integration Services, run the following Windows Installer files:

    • SSISSQL2014PDWDest-x86.msi

    • SSISSQL2014PDWDest-x64.msi

[TOP]

Step 4: Verify the Destination Adapters

To verify the installations

After the installation has finished, you can view your installed destination adapters in Control Panel, Programs and Features with the following names:

  • Microsoft SQL Server PDW SSIS Destination x86 Components

  • Microsoft SQL Server PDW SSIS Destination x64 Components

  • Microsoft SQL Server 2012 PDW SSIS Destination x86 Components

  • Microsoft SQL Server 2012 PDW SSIS Destination x64 Components

  • Microsoft SQL Server 2014 PDW SSIS Destination x86 Components

  • Microsoft SQL Server 2014 PDW SSIS Destination x64 Components

For SQL Server 2008 R2 Integration Services, continue to the next step to add SQL Server PDW as a data flow destination.

For SQL Server 2012 Integration Services, verify SQL Server PDW is listed as a destination in the SSIS toolbox in SQL Server Data Tools. To do this:

  1. Open SQL Server Data Tools for Visual Studio 2010. To open these, find the All Programs menu, select Microsoft SQL Server 2012, and select SQL Server Data Tools. To do this in:

    • Windows 7, open the Start menu, select All Programs, select Microsoft SQL Server 2012, and select SQL Server Data Tools.

    • Windows 8, press the Start key, select search, click Apps, find Microsoft SQL Server 2012. Under Microsoft SQL Server 2012, click to open SQL Server Data Tools. This opens SQL Server Data Tools for Visual Studio 2010.

  2. Open an existing Integration Services project or create a new one. At least one project must exist in order to add the data flow destination. To create a project:

    • Click on the File menu, choose New, choose Project, and click OK.

    • In the new project window, choose Integration Services Project under Business Intelligence. This opens the package design window.

    • In the Package.dtsx [Design] window, click on the Data Flow tab.

    • Scroll down through the SSIS Toolbox. You should now see SQL Server PDW Destination under Other Destinations in the SSIS Toolbox.

    • The SQL Server PDW Destination is installed correctly! For next steps, see Load Data With Integration Services (SQL Server PDW)

[TOP]

Step 5 – SQL Server 2008 R2 only: Add SQL Server PDW as a Data Flow Destination

Important note Important

This step is only required for SQL Server 2008 R2 Integration Services. Skip this step if you are using SQL Server 2012 Integration Services.

In this step, you will add SQL Server PDW as a Data Flow Destination in the Integration Services toolbox.

To add the SQL Server PDW Destination to the Toolbox

  1. Open SQL Server Business Intelligence Development Studio (BIDS). To do this in:

    • Windows 7, click on the Windows Start menu, select All Programs, select Microsoft SQL Server 2008 R2, and select SQL Server Business Intelligence Development Studio.

    • Windows 8, press the Start key, select search, click Apps, find Microsoft SQL Server 2008 R2. Under Microsoft SQL Server 2008 R2, click Business Intelligence Development Tools.

  2. Open an existing Integration Services project or create a new one. At least one project must exist in order to add the data flow destination. To create a project:

    • Click on the File menu, choose New, choose Project.

      Create a new Integration Services Project

    • In the new project window, choose Integration Services Project. This opens the package design window.

  3. Add SQL Server PDW Destination as a Data Flow Destination.

    • Select the Data Flow tab.

    • Scroll through the toolbox until you see the Data Flow Destinations category. Right-click on Data Flow Destinations, wait for the Choose Toolbox dialog to appear, and select Choose items …

    • In the Choose Items window, click the SSIS Data Flow Items tab.

    • Select SQL Server PDW Destination, and click OK.

      Check the SQL Server PDW box

    • You should now see SQL Server PDW Destination in the Data Flow Destinations of the Toolbox.

      SQL Server PDW is a Data Flow Destination

[TOP]

Next Steps

Your computer is now setup to build and run Integration Services loading packages. For more information, see Load Data With Integration Services (SQL Server PDW).

See Also

Concepts

How to create a script task that uses the SSIS PDW destination adapter (SQL Server PDW)


Install SQL Server Native Client (SQL Server PDW)

[image]

Install SQL Server Native Client (SQL Server PDW)
Send Feedback

This topic explains how to install the SQL Server Native Client onto the client that will connect to SQL Server PDW. SQL Server Native Client is required for many application connections to SQL Server PDW.

Parent Topic: Client Tools and Applications (SQL Server PDW)

Contents

  • Basics

  • Before You Begin

  • Step 1: Verify Your Current Installation

  • Step 2: Install SQL Server Native Client 10.0

  • Step 3: Install SQL Server Native Client 11.0

Basics

Microsoft SQL Server Native Client contains the OLE DB provider for SQL Server and the ODBC Driver for SQL Server, all in one native dynamic link library (DLL). SQL Server Native Client supports applications that use native-code APIs (ODBC, OLE DB and ADO) to communicate with SQL Server.

SQL Server Native Client is required for all ODBC and OLE DB application connections to SQL Server PDW. You can install SQL Server Native Client with SQL Server, or as a separate download package.

  • SQL Server 2008 R2 installs SQL Server Native Client 10.0.

  • SQL Server 2012 installs SQL Server Native Client 11.0.

When connecting with SQL Server Native Client, specify port 17001.

For more information, see these topics:

  • The SQL Server Native Client ... on MSDN.

  • Connection Strings for OLE DB and SQL Server Native Client (SQL Server PDW)

  • Connection Strings for ODBC and SQL Server Native Client (SQL Server PDW)

Before You Begin

Software Requirements

You can install SQL Server Native Client with SQL Server, or as a separate download package.

First, decide which version of SQL Server Native Client you will need in order to connect to SQL Server PDW. It’s a good idea to install both SQL Server Native Client 10.0 and SQL Server Native Client 11.0, if they are not already installed, to avoid not having the right version of SQL Server Native Client when you need it.

SQL Server PDW applications that work with SQL Server Native Client 10.0 or SQL Server Native Client 11.0

  • SQL Server PDW dwloader for loading data

  • SQL Server PDW dwsql Command-Line Tool

Microsoft applications that require SQL Server Native Client 10.0

  • SQL Server 2008 R2 sqlcmd

  • SQL Server 2008 R2 Business Intelligence Development Studio.

  • SQL Server 2008 R2 Integration Services

  • SQL Server 2008 R2 PowerPivot for Excel

  • SQL Server 2008 R2 Reporting Services

  • SQL Server 2008 R2 Analysis Services

  • SQL Server 2008 R2 Linked Server

Microsoft applications that require SQL Server Native Client 11.0

  • SQL Server 2012 sqlcmd

  • SQL Server 2012 SQL Server Data Tools

  • SQL Server 2012 Integration Services

  • SQL Server 2012 PowerPivot for Excel

  • SQL Server 2012 Reporting Services

  • SQL Server 2012 Analysis Services

  • SQL Server 2012 Linked Server

Step 1: Verify Your Current Installation

You might already have SQL Server Native Client 10.0 and SQL Server Native Client 11.0 installed. Use these instructions to verify if you have the correct versions installed.

Note Note

You need the 10.5 version of SQL Server Native Client 10.0. Using the ODBC Administrator (odbcad32.exe), the driver name shows as SQL Server Native Client 10.0, and the version shows as 2007.100.5500.00.

To verify the version of SQL Server Native Client

  • In Windows, click Start and enter odbcad32.exe in the search box.

  • In the ODBC Data Source Administrator Window click the System DSN tab.

  • Click the Drivers tab.

  • If you see SQL Server Native Client 10.0, verify the version is 2007.100.5500.00 or later. This is the version that ships with SQL Server 2008 R2. If the version is less than 2007.100.5500.00, you need to update the client.

    Version for SQL Server Native Client 10.0

  • If you see SQL Server Native Client 11.0, then it is already installed. Both 10.0 and 11.0 can be installed on the same system.

  • Close the ODBC Data Source Administrator window.

If the correct SQL Server Native Client is already installed, you don’t need to proceed any further with this topic.

Step 2: Install SQL Server Native Client 10.0

Install SQL Server Native Client 10.0 2007.100.5500.00 (or later), if it is not already installed, on the computer that is connecting to SQL Server PDW.

To install SQL Server Native Client 10.0 with SQL Server 2008 R2

  • Run SQL Server 2008 R2 setup and from the installer’s Features page, select “Client Connectivity Tools.”

To install SQL Server Native Client 10.0 from the Microsoft Download Center

Search on the internet to find the latest Microsoft SQL Server 2008 R2 Feature pack, and install SQL Server Native Client 10.0 from the links listed on the feature pack page.

For example, you can install SQL Server Native Client 10.0 from the Microsoft SQL Server 2008 R2 SP1 Feature Pack, published 7/11/2011 on the Microsoft Download Center.

  • For x86 computers, install 1033\x86\sqlncli.msi.

  • For x64 computers, install 1033\x64\sqlncli.msi.

For more information, see Installing SQL Server Native Client on MSDN.

Step 3: Install SQL Server Native Client 11.0

Install SQL Server Native Client 11.0, if it is not already installed, on the computer that is connecting to SQL Server PDW.

To install SQL Server Native Client 11.0 with SQL Server 2012

  • Run SQL Server 2012 setup and from the installer’s Features page, select “Client Connectivity Tools.”

To install SQL Server Native Client 11.0 from the Microsoft Download Center

Search on the internet to find the latest Microsoft SQL Server 2012 Feature pack, and install SQL Server Native Client 11.0 from the links listed on the feature pack page.

For example, you can install SQL Server Native Client 11.0 from the Microsoft SQL Server 2012 SP1 Feature Pack, published 11/13/2012 on the Microsoft Download Center.

  • For x86 computers, install ENU\x86\sqlncli.msi.

  • For x64 computers, install ENU\x64\sqlncli.msi.

For more information, see Installing SQL Server Native Client on MSDN.


Install dwloader Command-Line Loader (SQL Server PDW)

[image]

Install dwloader Command-Line Loader (SQL Server PDW)
Send Feedback

Describes how to install the dwloader Command-Line Loader tool for SQL Server PDW.

Parent Topic: Client Tools and Applications (SQL Server PDW)

Before You Begin

The client tools for SQL Server PDW ship with the appliance. Ask your appliance administrator about how to obtain the installation files for the tools, or see Client Tools and Applications (SQL Server PDW) for the location of the tools..

To install dwloader, you will need the ClientTools.msi file. This installs dwloader and dwsql.

Install dwloader and dwsql

ClientTools.msi is a Windows Installer file that installs dwloader Command-Line Loader (SQL Server PDW) and Connect With dwsql Command-Line SQL Client (SQL Server PDW).

You can install the ClientTools.msi file from the UI or from the command-line. The command-line has an option called CONTROL_NODE_ADDRESS="control_node_name" that sets the default for the dwloader –S option. This sets the default appliance that will receive the loaded data. control_node_name is a string that specifies the name of the Control node according to the name in the appliance DNS server. For more

For example, the following command installs dwloader and dwsql, and sets the default for the dwloader –S option to PDW-CTLN. The value "PDW-CTLN" is the name of the Control node as stored stored in the appliance DNS server.

Copy image Copy Code
Msiexec /i ClientTools.msi CONTROL_NODE_ADDRESS="PDW-CTLN"

When running msiexec, the standard options are available. To learn more, run the command msiexec /?


Install AdventureWorksPDW2012 (SQL Server PDW)

[image]

Install AdventureWorksPDW2012 (SQL Server PDW)
Send Feedback

This topic describes how to install the AdventureWorksPDW2012 sample database onto SQL Server PDW.

Parent Topic: Client Tools and Applications (SQL Server PDW)

Contents

  • Before You Begin

  • Step 1: Obtain the Installation File

  • Step 2: Install the Sample Database

  • Step 3: Verify the Database is Installed

Before You Begin

Install the dwloader loading tool, if it is not already installed, onto the computer that will run the AdventureWorksPDW2012 installation scripts. For installation instructions, see Install dwloader Command-Line Loader (SQL Server PDW).

Step 1: Obtain the Installation File

Obtain the AdventureWorksPDW2012.zip file from your appliance administrator, or see Client Tools and Applications (SQL Server PDW) for location information. This file contains the scripts and data that you need to install the AdventureWorksPDW2012 sample database onto SQL Server PDW.

Step 2: Install the Sample Database

To install the AdventureWorksPDW2012 sample database:

  1. Open Windows Explorer and navigate to the folder that contains AdventureWorksPDW2012.zip.

  2. Right-click on the AdventureWorksPDW2012.zip file, and choose Extract All. This will extract the files to an AdventureWorksPDW2012 folder.

  3. Edit aw_create.bat and set the following values:

    • server = IP address of the Control node cluster

    • user = SQL Server PDW login that has permission to load data and create a database.

    • password = User password.

  4. Save your changes to aw_create.bat.

  5. Right-click the aw_create.bat file, and select Open. Alternatively, to see the command-line activity, you can run the aw_create.bat file from a command-prompt.

Step 3: Verify the database is installed

To verify the installation is complete, open SQL Server Data Tools and view the databases on your appliance. You should see AdventureWorksPDW2012 in the list of databases.


Install sqlcmd Command-Line Client (SQL Server PDW)

[image]

Install sqlcmd Command-Line Client (SQL Server PDW)
See Also Send Feedback

Describes how to install the sqlcmd Command-Line Client for use with SQL Server PDW. Use these instructions if you do not already have SQL Server installed and need to install the standalone version of sqlcmd.

Note Note

If you already have SQL Server 2008 R2, SQL Server 2012, or SQL Server 2014, you already have sqlcmd and the correct version of SQL Server Native Client, unless you unchecked Client Tools during setup.

Sqlcmd, when installed with SQL Server, is located in these directories:

  • SQL Server 2014: Client SDK\ODBC\110\Tools\Binn

  • SQL Server 2012: Client SDK\ODBC\110\Tools\Binn

  • SQL Server 2008 R2: Client SDK\ODBC\100\Tools\Binn

Contents

  • Before You Begin

  • Install sqlcmd

  • Next Steps

Before You Begin

To install SQL Server Native Client, see Install SQL Server Native Client (SQL Server PDW).

Software Prerequisites - SQL Server 2014

SQL Server Native Client 11.0

SQL Server 2014 ships with the SQL Server 2012 version of sqlcmd.

Software Prerequisites - SQL Server 2012

  • SQL Server Native Client 11.0

Software Prerequisites - SQL Server 2008 R2

  • SQL Server Native Client 10.0 (release 10.5)

[TOP]

Install sqlcmd

To install sqlcmd as a standalone package, use the following download links. These links are also listed on the Microsoft SQL Server 2012 Feature Pack page on MSDN.

  • X86 Package (SqlCmdLnUtils.msi)

  • X64 Package (SqlCmdLnUtils.msi)

[TOP]

Next Steps

To run sqlcmd with SQL Server Native Client 11.0, use the –I option.

For more information on using sqlcmd with SQL Server PDW, see Connect With sqlcmd Command-Line SQL Client (SQL Server PDW).

For the sqlcmd reference for SQL Server 2012, see sqlcmd Utility.[TOP]

See Also

Other Resources

Client Tools (SQL Server PDW)


Connect With Applications (SQL Server PDW)

[image]

Connect With Applications (SQL Server PDW)
See Also Send Feedback

Provides a list of the topics for connecting to SQL Server PDW with applications.

Topics in this Section

  • Connect With PowerPivot for Excel (SQL Server PDW)

  • Connect With SQL Server Integration Services for Querying (SQL Server PDW)

  • Connect With SQL Server Integration Services for Loading (SQL Server PDW)

  • Connect With SQL Server Reporting Services (SQL Server PDW)

  • Connect With SQL Server Analysis Services (SQL Server PDW)

  • Connect With SQL Server Linked Server (SQL Server PDW)

  • Connect With SQL Server Report Builder (SQL Server PDW)

  • Connect With sqlcmd Command-Line SQL Client (SQL Server PDW)

  • Connect With dwsql Command-Line SQL Client (SQL Server PDW) – deprecated.

Related Tasks for Connecting With Applications

The following table provides examples of connecting to SQL Server PDW with applications. For each application, the table gives a link to an application-specific topic that gives the requirements and instructions for connecting to SQL Server PDW.

Task to Accomplish | Application to Use | Description
Use a graphical SQL query client. | SQL Server Data Tools | Install SQL Server database tooling for Visual Studio (SQL Server PDW)
Use a command-line client to query SQL Server PDW. | sqlcmd | Connect With sqlcmd Command-Line SQL Client (SQL Server PDW)
Load data with Integration Services. | Integration Services | Install Integration Services Destination Adapters (SQL Server PDW) Load Data With Integration Services (SQL Server PDW)
Quickly analyze high volumes of data within a spreadsheet. | PowerPivot For Excel | Connect With PowerPivot for Excel (SQL Server PDW).
Perform data mining and other analysis tasks. | Analysis Services | Connect With SQL Server Analysis Services (SQL Server PDW)
Generate reports using SQL Server PDW data | Reporting Services | Connect With SQL Server Reporting Services (SQL Server PDW)
Use SQL Server to submit SQL statements to SQL Server PDW. | Linked Server. | Connect With SQL Server Linked Server (SQL Server PDW)

[TOP]

See Also

Other Resources

Client Tools (SQL Server PDW)


Connect with Power BI or Power Query by using Data Management Gateway (Analytics Platform System)

[image]

Connect with Power BI or Power Query by using Data Management Gateway (Analytics Platform System)
Send Feedback

Connect to Analytics Platform System with Power BI by using an OData feed from the Microsoft Data Management Gateway.

Create and use a PDW data source in Power BI

These instructions show you how to use Power BI to bring an OData feed into Excel from a PDW database.

The full documentation for creating a PowerBI data source is at Create a Data Source and Enable OData Feed in Power BI Admin Center. This instructions give specifics about using an OData feed from a PDW database.

Before you begin

  • This walkthrough requires that a secure gateway is already configured on PDW. If this is not the case, contact Microsoft Support for assistance.

To create a PDW data source

  1. Go to the Power BI Admin Center and verify the Data Management Gateway is running. To do this:

    • Open Power BI for Office 365 and sign in.

    • Click the Power BI tile.

    • Click the cog in the upper right corner and select Power BI Admin Center.

    • On the gateways page, verify that the status of your gateway is Running.

  2. Click data sources in the left sidebar, select Add New Data Source and choose SQL Server.

  3. Check the Enable Cloud Access and Enable OData Feed options. Click next.

  4. In the connection info window, fill in these fields:

    • Name: Enter the descriptive name for your data source.

    • Description: Optional description of the data source.

    • Gateway: Select your installed gateway.

    • Data Source Type: Enter SQL Server.

    • Connect using: Choose Connection Properties.

    • Connection provider: Select SQL Server Native Client 11.0

    • Server name: Enter the PDW Control node IP, comma, 17001. For example, 10.10.10.10,17001.

    • Database: Select your PDW source database for the OData feed.

    Click set credentials to launch a wizard that runs locally on your client.

  5. The wizard will present a data source settings window. Fill in these fields:

    • Credentials type: Select Database.

    • User name, Password: Enter the PDW user name and password.

    • Encrypt connection: No checkmark, leave this unchecked.

    • Privacy level: Select the appropriate privacy level.

    Click test connection.

    Click OK. The wizard will take you to the data settings.

  6. In data settings, select all the database tables to expose in the OData feed. Click next.

  7. In users and groups, add all the users and groups that should have access to the data source. Click finish. This completes the wizard.

  8. Get the OData feed URL. To do this, in the PowerBI Admin Center click on the three dots listed with your data source. You will need this in Power Query to consume data from the data source.

To use the OData feed in Power Query

  1. Open Excel and click on the POWER QUERY tab. If this tab doesn’t exist, you need to first install Power Query.

  2. Click the From Other Sources drop down menu, and select From Odata Feed.

  3. Enter the OData feed URL for your data source. Click OK.

  4. You should now see your PDW database in the Navigator sidebar.

  5. Click Load to populate the Excel worksheet with the table data.

Create a Power Query data source for data refresh in Power BI

Before you can schedule a data refresh, you need to configure all data sources in your Power Query connection. Repeat these steps for each data source.

The OData feed connection string can’t be used for the data refresh scenario. To get a proper connection string, we will show how to import data with a SQL Server connection and then use the connection properties to get the data refresh connection string. Your Excel workbook need to have a data model in it to enable the data refresh scenario.

To get a Power Query connection string for data refresh

  1. Import data using a SQL Server data source. To do this, open Excel, click the POWER QUERY tab, click the From Database menu, and select From SQL Server Database.

  2. In the Microsoft SQL Database dialog, enter this information:

    • Server: PDW Control node IP, comma (,), and port 17001. For example, 10.10.10.10,17001.

    • Database (optional): Enter your database, and click OK.

  3. The wizard will present a data source settings window. Fill in these fields:

    1. Credentials type: Select Database.

    2. User name, Password: Enter the PDW user name and password.

    3. Encrypt connection: No checkmark, leave this unchecked.

    4. Privacy level: Select the appropriate privacy level. For more information, see Configure a privacy level.

    Click test connection and verify the connection status is Configured with a green checkmark. Click Next.

  4. Excel will populate the Navigator with names of all the tables present in the database. In the Navigator, check Select multiple items, check the tables you want to load, check Load to worksheet, check Load to Data Model, and click Load.

  5. Excel will now populate the table data and create a data model.

  6. You can now get the connection string. To do this:

    • Click the Data tab and click Connections.

    • Select any worksheet.

    • In the Workbook Connections dialog, click Properties.

    • In the Connection Properties dialog, click Definition.

    • The Connection string text box contains the connection string that you can use for data refresh. Copy this and save it for data refresh configuration.

To configure a Power Query data source

  1. To configure the data source, you will need the correct Power Query connection string. For information about how to obtain this, see Scheduled Data Refresh for Power Query and Create a Data Source and Enable Cloud Access.

  2. Go to the PowerBI Admin Center) and make sure your gateway is running.

  3. In the data sources pane, click new data source and select Power Query.

  4. In the connection info page, enter your Power Query connection string. Click next.

  5. In the data source info page, enter this information:

    • Name: A user-friendly name for the data source. This will be used to identify the data source in the Power BI Admin Center.

    • Description: Optional description for the data source.

    • Gateway: Select a gateway to host the data source.

    • Connection provider: Select .Net Framework Data Provider for SQL Server.

    • Connection string: Enter your connection string if it is not there already.

    Click credentials to launch a wizard that runs locally on your client.

  6. The wizard will present a data source settings window. Fill in these fields:

    • Credentials type: Select Database.

    • User name, Password: Enter the PDW user name and password.

    • Encrypt connection: No checkmark, leave this unchecked.

    • Privacy level: Select the appropriate privacy level. For more information, see Configure a privacy level.

    Click test connection and verify the connection status is Configured with a green checkmark. Click Next.

  7. In the users and groups page, add users who can access the Power Query connection. Click save.

To schedule a Power Query data refresh

  1. In Office Online, click Power BI and then select team site.

  2. Upload your Excel workbook that you used to import the data. To do this, click add and select Upload file. Browse to your file and upload it.

  3. To schedule the data refresh, locate your Excel workbook, click the three dots, and select Schedule Data Refresh.

  4. In the Data refresh wizard, slide the rule to enable Refresh.

  5. Select the workbooks to refresh, configure the refresh schedule, and enable notification in the respective settings.

  6. Save the settings. You can save and refresh the report.

Performance Considerations

As the number of concurrent requests for data through the Data Management Gateway increases, the response time can slow down. We recommend scheduling data refreshes during off-peak appliance usage hours to reduce the impact on PDW query execution time.

The Data Management Gateway consumes CPU and memory resources on the appliance. Hence, there is a limit for the number of OData feeds that can run concurrently.

Limitations and Restrictions

The OData protocol requires that at least one column has a NOT NULL constraint. Without this constraint, the table will not be available for OData.


Connect With PowerPivot for Excel (SQL Server PDW)

[image]

Connect With PowerPivot for Excel (SQL Server PDW)
See Also Send Feedback

This topic describes how to connect to SQL Server PDW with PowerPivot for Excel. PowerPivot for Excel is a free download that significantly expands the data analysis capabilities of Excel.

Contents

  • Before You Begin

  • Connect with PowerPivot for Excel

Before You Begin

Software Prerequisites

SQL Server Native Client 10.0. See Install SQL Server Native Client (SQL Server PDW)

.NET Framework 3.5 SP1 or higher. For installation details, see the Microsoft .NET Framework 3.5 Service Pack 1 page on the Microsoft Download Center. If you have Windows 7, you already have this.

Microsoft Excel 2010 (64-bit recommended) with the PowerPivot add-in installed. For installation instructions, see PowerPivot: Install the PowerPivot Add-In for Excel on Technet.

TOP

Connect With PowerPivot for Excel

The following steps explain how to connect by using ADO.NET and the .NET Framework Data Provider for SQL Server (SqlClient).

To connect, follow these steps:

  1. Launch Excel. You must be using Excel 2010, and must have the PowerPivot add-on installed.

  2. Click the PowerPivot tab.

  3. Click PowerPivot Window Launch.

  4. On the Home tab,

  5. On the Home tab, in the Get External Data options, click From Database, and select From SQL Server.

    Choose Get External Data from Other Sources

  6. In the Table Import Wizard,

    • For Friendly connection name, enter a connection name.

    • For User name and Password fields, enter your SQL Server PDW login and password.

    • In the Database Name drop-down, select the name of the target database.

    • For Server name, enter the IP address of the Control node cluster, followed by a comma (,), and port 17001. For example: 10.192.63.147,17001 .

    • For Log on to the server, choose Use Windows Authentication, or choose Use SQL Server Authentication and enter your login and password.

    • Click Next to connect.

  7. The connection to SQL Server PDW is finished. Continue through the Wizard to select the rows to import.

TOP

See Also

Concepts

Connect With Applications (SQL Server PDW)


Connect With SQL Server Integration Services for Querying (SQL Server PDW)

[image]

Connect With SQL Server Integration Services for Querying (SQL Server PDW)
See Also Send Feedback

This topic describes how to create an Integration Services connection for querying SQL Server PDW.

Contents

  • Connect With SQL Server 2008 R2 Integration Services

  • Connect with SQL Server 2012 Integration Services

  • Connect with SQL Server 2014 Integration Services

Connect With SQL Server 2008 R2 Integration Services

Before You Begin

Permissions

To query data on SQL Server PDW, you need a SQL Server PDW login that has read permission on the data you plan to query.

Software Prerequisites

  • SQL Server 2008 R2 Integration Services, standard or enterprise edition.

  • SQL Server Business Intelligence Development Studio (BIDS). See Introducing Business Intelligence Development Studio.

  • .NET Framework 3.5 SP1 or higher. See Microsoft .NET Framework 3.5 Service Pack 1. If you have Windows 7 or later, you already have this.

  • SQL Server Native Client (SNAC) 10.0. See Install SQL Server Native Client (SQL Server PDW).

Create an Integration Services Project

Create an Integration Services project by using Business Intelligence Development Studio. To do this:

  1. Open Business Intelligence Development Studio. To do this, in Windows click Start, choose Programs, choose SQL Server 2008 R2, and then select SQL Server Business Intelligence Development Studio.

  2. Open a new project. To do this, on the File menu, select New, and choose Project.

  3. In the New Project window, for Project type, select Business Intelligence Projects, and then select the template, Integration Services Project. Enter a project name and solution name, and click OK.

Analysis Services 2008 R2 New Project

Add a Data Source

To add a data source:

  1. In Solution Explorer, under your new project, right-click Data Sources, and select New Data Source.

  2. Click Next to connect.

  3. In the Welcome to the Data Source Wizard screen, click Next.

    Data Source Wizard

  4. In the Select how to define the connection screen, choose Create a data source based on an existing or new connection, and click New.

  5. In the Connection Manager window, make the following selections:

    • Provider: Select Native OLE DB\SQL Server Native Client 10.0.

    • Server name for Ethernet connections: enter the IP address of the Control node cluster, followed by a comma (,), followed by port 17001. For example, 10.192.63.147,17001 .

      Server name for InfiniBand connections: enter <appliance-name>-SQLCTL01,17001

    • Choose Use Windows Authentication, or choose Use SQL Server Authentication.

    • User name and Password fields: enter your SQL Server PDW login and password if using SQL Server Authentication.

    • Select or enter a Database Name: Select or enter the name of the target database.

      Analysis Services Connection

  6. Click Test Connection to verify that the new data source is available.

  7. Click OK to return to the page, Select how to define the connection.

  8. In the Select how to define the connection window, view your new connection listed under Data Connections.

  9. Click Finish to bypass the Impersonation Information page.

  10. On the Completing the Wizard page, type a friendly name for your data source.

  11. You can now see your data source listed in Solution Explorer under your Integration Services Project.

    Data source visible in BIDS Solution Explorer

Connect With SQL Server 2012 Integration Services

Before You Begin

Software Prerequisites for SQL Server 2012 Integration Services

  • SQL Server 2012 Integration Services, standard or enterprise edition.

  • SQL Server Data Tools. These are already installed unless you unchecked Client Connectivity Tools in your SQL Server 2012 installation.

  • .NET Framework 3.5 SP1 or higher. See Microsoft .NET Framework 3.5 Service Pack 1. If you have Windows 7 or later, you already have this.

  • SQL Server Native Client (SNAC) 11.0. If you have SQL Server 2012, you already have this unless you unchecked Client Connectivity Tools during SQL Server 2012 setup. See Install SQL Server Native Client (SQL Server PDW).

Create an Integration Services Project

To create an Integration Services Project:

  1. Open SQL Server Data Tools from your SQL Server 2012 installation. This opens the Visual Studio 2010 Integrated Shell.

    • On Windows 7, click on the Windows Start menu, select All Programs, select Microsoft SQL Server 2012, and select SQL Server Data Tools.

    • On Windows 8, press the Start key, select search, click Apps, find Microsoft SQL Server 2012. Under Microsoft SQL Server 2012, click SQL Server Data Tools.

    SQL Server Data Tools

  2. Open an existing Integration Services project or create a new one. To create a project:

    • Click on the File menu, choose New, choose Project.

      New SQL Server 2012 Integration Services Project

    • In the new project window, choose Integration Services Project and click OK.

Add a Data Source

To add a data source:

  1. In Solution Explorer, under your new project, right-click Data Sources, and select New Data Source.

  2. Click Next to connect.

  3. In the Welcome to the Data Source Wizard screen, click Next.

    Data Source Wizard

  4. In the Select how to define the connection screen, choose Create a data source based on an existing or new connection, and click New.

  5. In the Connection Manager window, make the following selections:

    • Provider: Select Native OLE DB\SQL Server Native Client 10.0.

    • Server name for Ethernet connections: enter the IP address of the Control node cluster, followed by a comma (,), followed by port 17001. For example, 10.192.63.147,17001 .

      Server name for InfiniBand connections: enter <appliance-name>-SQLCTL01,17001

    • Choose Use Windows Authentication, or choose Use SQL Server Authentication.

    • User name and Password fields: enter your SQL Server PDW login and password if using SQL Server Authentication.

    • Select or enter a Database Name: Select or enter the name of the target database.

      Analysis Services Connection

  6. Click Test Connection to verify that the new data source is available.

  7. Click OK to return to the page, Select how to define the connection.

  8. In the Select how to define the connection window, view your new connection listed under Data Connections.

  9. Click Finish to bypass the Impersonation Information page.

  10. On the Completing the Wizard page, type a friendly name for your data source.

  11. You can now see your data source listed in Solution Explorer under your Integration Services Project.

    Data source visible in BIDS Solution Explorer

Connect With SQL Server 2014 Integration Services

Before You Begin

Software Prerequisites for SQL Server 2014 Integration Services

  • SQL Server 2014 Integration Services, standard or enterprise edition.

  • SQL Server Data Tools. These are already installed unless you unchecked Client Connectivity Tools in your SQL Server 2014 installation.

  • .NET Framework 3.5 SP1 or higher. See Microsoft .NET Framework 3.5 Service Pack 1. If you have Windows 7 or later, you already have this.

  • SQL Server Native Client (SNAC) 12.0. If you have SQL Server 2014, you already have this unless you unchecked Client Connectivity Tools during SQL Server 2014 setup. See Install SQL Server Native Client (SQL Server PDW).

See Also

Other Resources

Client Tools (SQL Server PDW)


Connect With SQL Server Integration Services for Loading (SQL Server PDW)

[image]

Connect With SQL Server Integration Services for Loading (SQL Server PDW)
See Also Send Feedback

This topic describes how to create an Integration Services package that connects to SQL Server PDW for the purpose of loading data.

Contents

  • Connect with SQL Server 2008 R2 Integration Services

  • Connect with SQL Server 2012 Integration Services

Connect with SQL Server 2008 R2 Integration Services

Before You Begin

Permissions

You will need permission to connect to SQL Server PDW in order to test the connection.

Software Prerequisites

  • SQL Server Business Intelligence Development Studio (BIDS). See Introducing Business Intelligence Development Studio.

  • .NET Framework 3.5 SP1 or higher. For installation details, see Microsoft .NET Framework 3.5 Service Pack 1. If you have Windows 7 or later, you already have this.

  • SQL Server Native Client (SNAC) 10.0 for SQL Server 2008 R2. See Install SQL Server Native Client (SQL Server PDW).

  • SQL Server 2008 R2 Integration Services Destination Adapters for SQL Server PDW, both 32-bit and 64-bit. See Install Integration Services Destination Adapters (SQL Server PDW).

Create an Integration Services project

To create an Integration Services project:

  1. Open SQL Server Business Intelligence Development Studio (BIDS). To do this:

    • On Windows 7, click on the Windows Start menu, select All Programs, select Microsoft SQL Server 2008 R2, and select SQL Server Business Intelligence Development Studio.

    • On Windows 8, press the Start key, select search, click Apps, find Microsoft SQL Server 2008 R2. Under Microsoft SQL Server 2008 R2, click Business Intelligence Development Tools.

  2. Open an existing Integration Services project or create a new one. At least one project must exist in order to add the data flow destination. To create a project:

    • Click on the File menu, choose New, choose Project.

      Create a new Integration Services Project

    • In the new project window, choose Integration Services Project. This opens the package design window.

Add a data flow task that has a SQL Server PDW Destination

  1. Add a data flow task to the Control Flow items.

    • In the Design window, click on the Control Flow tab.

    • In the Toolbox, under Control Flow Items, left-click the Data Flow Task and drag it into the Control Flow pane.

  2. Add SQL Server PDW as the destination for the data flow task.

    • In the Control Flow window, double-click the data flow task. This opens the Data Flow tab.

    • In the Toolbox, under Data Flow Destinations, left-click the SQL Server PDW Destination and drag it into the Data Flow pane. The SQL Server PDW destination will appear in the Data Flow pane.

    OLE DB data flow task

Connect the SQL Server PDW Destination to Your Appliance

Configure the SQL Server PDW Destination to connect to your appliance. To do this:

  1. Double-click the SQL Server PDW data flow task to open the SQL Server PDW Destination Editor.

  2. In the Connection Manager field, select <Create new connection>.

  3. In the SQL Server PDW Connection Manager Editor window, enter the following on the General tab:

    • Server name for Ethernet connections: enter the IP address of the Control node cluster, followed by a comma (,), followed by port 17001. For example, 10.192.63.147,17001 .

      Server name for InfiniBand connections: enter <appliance-name>-SQLCTL01,17001

    • For User and Password, enter the SQL Server PDW login and password that has permissions to load data.

    • For Destination database name, enter the target database on SQL Server PDW.

    • For Staging database name, enter the name of the staging database. This field must be empty for fastappend loading mode, and is optional for other load modes. For more information, see Create the Staging Database (SQL Server PDW).

      SQL Server 2008 R2 Integration Services

      Your appliance connection information

    • Click OK to return to the Destination Editor window.

    • Select your destination table.

    • Verify the loading mode. If the loading mode is FastAppend, uncheck the Roll-back load on table update or insert failure checkbox. Click OK.

      • By unchecking the box, loads from the staging table to a distributed destination table are performed and committed in parallel across all distributions. This performs faster than serializing the loads, but is not transaction-safe. This is equivalent to using the –m option in dwloader.

      • By checking the box, loads from the staging table to a distributed destination table are performed serially across the distributions within each Compute node, and in parallel across the Compute nodes. This performs slower than parallelizing the loads, but is transaction-safe. This is equivalent to not using the –m option in dwloader.

  4. Specify the source for the data flow.

    • Click the Data Flow tab and then drag a data source to the Data Flow pane. For example, if your data source is in a flat file, drag the Flat File Source to the Data Flow pane.

    • Double-click the box you just created. This opens the Flat File Source Editor.

    • Click Connection Manager and then click New to create a new connection.

    • In the Connection manager name box, enter a friendly name for the connection.

    • Click Browse and select the file to use as the data source.

    • Specify formatting information by using the options in the Flat File Source Editor.

  5. Specify the data flow from source to destination.

    • On the Data Flow pane, drag the green arrow from the Flat File Source box to the SQL Server PDW Destination box.

    • Map the columns by dragging to Input and Destination.

    • Save the project.

  6. Optionally, add transformations to filter, combine, and cleanse the data before you load it into the data warehouse.

  7. Save the package. The project gets saved to a .dtsx file.

  8. Save your changes and exit.

[TOP]

Connect with SQL Server 2012 Integration Services

Before You Begin

Permissions

To create the Integration Services package, you need permission to connect to SQL Server PDW in order to test the connection.

Software Prerequisites

  • SQL Server Data Tools. To install the required tools, install the Client Connectivity Tools in your SQL Server 2012 installation. You already have this installed unless you unchecked Client Connectivity Tools in your SQL Server 2012 installation.

  • .NET Framework 3.5 SP1 or higher. See Microsoft .NET Framework 3.5 Service Pack 1. If you have Windows 7 or later, you already have this.

  • SQL Server Native Client (SNAC) 11.0. If you have SQL Server 2012, you already have this unless you unchecked Client Connectivity Tools during SQL Server 2012 setup. See Install SQL Server Native Client (SQL Server PDW).

  • SQL Server 2012 Integration Services Destination Adapters for SQL Server PDW, both 32-bit and 64-bit. See Install Integration Services Destination Adapters (SQL Server PDW).

[TOP]

Create an Integration Services Project

To create an Integration Services Project:

  1. Open SQL Server Data Tools from your SQL Server 2012 installation. This opens the Visual Studio 2010 Integrated Shell.

    • On Windows 7, click on the Windows Start menu, select All Programs, select Microsoft SQL Server 2012, and select SQL Server Data Tools.

    • On Windows 8, press the Start key, select search, click Apps, locate Microsoft SQL Server 2012. Under Microsoft SQL Server 2012, click SQL Server Data Tools.

    SQL Server Data Tools

  2. Open an existing Integration Services project or create a new one. To create a project:

    • Click on the File menu, choose New, choose Project.

      New SQL Server 2012 Integration Services Project

    • In the new project window, choose Integration Services Project and click OK.

Add a data flow task that has a SQL Server PDW Destination

  1. Add a data flow task to the Control Flow pane.

    • In the Package.dtsx [Design] window, click on the Control Flow tab.

    • In the SSIS Toolbox, left-click the Data Flow Task and drag it into the Control Flow pane.

  2. Add SQL Server PDW as the destination for the data flow task.

    1. In the Package.dtsx [Design] window, click on the Data Flow tab.

    2. In the SSIS Toolbox, under Other Destinations, left-click the SQL Server PDW Destination and drag it into the Data Flow pane.

      Data Flow Tab

Connect the SQL Server PDW Destination to Your Appliance

Configure the SQL Server PDW Destination to connect to your appliance. To do this:

  1. Double-click the SQL Server PDW data flow task to open the SQL Server PDW Destination Editor.

  2. In the Connection Manager field, select <Create new connection>.

  3. In the SQL Server PDW Connection Manager Editor window, enter the following on the General tab:

  4. For InfiniBand connections, Server name: Enter <appliance-name>-SQLCTL01,17001.

    For Ethernet connections, Server name: Enter the IP address of the Control node cluster, comma, port 17001. For example, 10.192.63.134,17001.

    User and Password: Enter the SQL Server PDW login and password that has permissions to load data.

    Destination database name: Use the drop down menu to select the target database on SQL Server PDW.

    Staging database name: Use the drop down menu to select the staging database. This field must be empty for fastappend loading mode, and is optional for other load modes. For more information, see Create the Staging Database (SQL Server PDW).

  5. Test the connection and click OK to close the connection success message. Click OK to close the SQL Server PDW Connection Manager and return to the Destination Editor window.

  6. Destination table: Use the drop down menu to select your destination table.

  7. Verify the loading mode. If the loading mode is FastAppend, uncheck the Roll-back load on table update or insert failure checkbox. Click OK.

    • By unchecking the box, loads from the staging table to a distributed destination table are performed and committed in parallel across all distributions. This performs faster than serializing the loads, but is not transaction-safe. This is equivalent to using the –m option in dwloader.

    • By checking the box, loads from the staging table to a distributed destination table are performed serially across the distributions within each Compute node, and in parallel across the Compute nodes. This performs slower than parallelizing the loads, but is transaction-safe. This is equivalent to not using the –m option in dwloader.

    Integration Services Connection to PDW

  8. Choose your Error Handling method.

  9. Leave the column mapping for later when you have defined a data source.

  10. Click OK to return to the Data Flow Task.

  11. Save the package.

[TOP]

Next Steps

The connection to SQL Server PDW is configured. You are now ready to load data with Integration Services. See Load Data With Integration Services (SQL Server PDW)

[TOP]

See Also

Other Resources

Client Tools (SQL Server PDW)


Connect With SQL Server Reporting Services (SQL Server PDW)

[image]

Connect With SQL Server Reporting Services (SQL Server PDW)
See Also Send Feedback

This topic describes how to connect to SQL Server PDW Version 2 from SQL Server 2014 Reporting Services and from SQL Server 2008 R2 Reporting Services.

Contents

  • Before You Begin

  • Create a Data Source (SQL Server 2008 R2 Reporting Services)

  • Create a Data Source (SQL Server 2012 Reporting Services)

Before You Begin

Software Prerequisites for SQL Server 2008 R2 Reporting Services

  • SQL Server 2008 R2 Reporting Services

  • Business Intelligence Development Studio (BIDS). For more information, see Introducing Business Intelligence Development Studio.

  • SQL Server Native Client (SNAC) 10.0. For installation instructions, see Install SQL Server Native Client (SQL Server PDW).

Software Prerequisites for SQL Server 2012 Reporting Services

  • SQL Server 2012 Reporting Services

  • SQL Server Data Tools (SSDT). For more information, see Reporting Services in SQL Server Data Tools (SSDT).

  • SQL Server Native Client (SNAC) 11.0. For installation instructions, see Install SQL Server Native Client (SQL Server PDW).

[TOP]

Create a Data Source (SQL Server 2008 R2 Reporting Services)

To create a Reporting Services data source that connects to your SQL Server PDW appliance

  1. Open SQL Server Business Intelligence Studio. To do this, in Windows click Start, choose All Programs, choose SQL Server 2008 R2 , and then select SQL Server Business Intelligence Development Studio.

  2. Open a new Reporting Services project. To do this, on the File menu, select New, and choose Project.

  3. In the New Project window, for Project type, select Business Intelligence Projects, and then select the template, Report Server Project. Click OK.

    New Reporting Services Project

  4. In Solution Explorer, you will see your new Report project. Under your project, right-click the Shared Data Sources folder, and select Add New Data Source.

  5. In the Shared Data Source Properties dialog box, type a name for the new data source. For type, select Microsoft SQL Server.

  6. Click Edit to create the connection string.

    Reporting Services Data Source Properties

  7. In the Connection Properties window, make the following selections:

    • Data source: Microsoft SQL Server (SqlClient)

    • Server name: Enter the IP address of the Control node cluster, followed by a comma, followed by the port name of 17001. For example 10.192.63.147,17001.

    • Choose Use Windows Authentication, or choose Use SQL Server Authentication.

    • User name and Password fields: enter your SQL Server PDW login and password if using SQL Server Authentication.

    • Select or enter a Database Name: Select or enter the name of the target database.

      Report Builder 3.0 Connection Properties

  8. Click Test Connection to verify that the new data source is available.

  9. Click OK.

  10. You can now view your connection string in the Data Source Properties windows.

    Reporting Services Shared Data Source Properties

  11. Click OK to return to the Report Builder window.

  12. In the Solution Explorer pane, view your new connection listed under Shared Data Sources.

    New Reporting Services data source

[TOP]

Create a Data Source (SQL Server 2012 Reporting Services)

To create a Reporting Services data source that connects to your SQL Server PDW appliance

  1. Open SQL Server Data Tools (SSDT). To do this, in Windows click Start, choose All Programs, choose SQL Server 2012, and then select SQL Server Data Tools.

  2. Open a new Reporting Services project. To do this, on the File menu, select New, and choose Project.

  3. In the New Project window, click Business Intelligence, click Reporting Services, click Report Server Project, and then click OK.

  4. In Solution Explorer, you will see your new Report project. Under your project, right-click the Shared Data Sources folder, and select Add New Data Source.

  5. In the Shared Data Source Properties dialog box, type a name for the new data source. For type, select Microsoft SQL Server.

  6. Click Edit to create the connection string.

    Reporting Services Data Source Properties

  7. In the Connection Properties window, make the following selections:

    • Data source: Microsoft SQL Server (SqlClient)

    • Server name: Enter the IP address of the Control node cluster, followed by a comma, followed by the port name of 17001. For example 10.192.63.147,17001.

    • Choose Use Windows Authentication, or choose Use SQL Server Authentication.

    • User name and Password fields: enter your SQL Server PDW login and password if using SQL Server Authentication.

    • Select or enter a Database Name: Select or enter the name of the target database.

      Report Builder 3.0 Connection Properties

  8. Click Test Connection to verify that the new data source is available.

  9. Click OK.

  10. You can now view your connection string in the Shared Data Source Properties windows.

    Reporting Services Shared Data Source Properties

  11. Click OK to return to the Report Builder window.

  12. In the Solution Explorer pane, view your new connection listed under Shared Data Sources.

    New Reporting Services data source

[TOP]

See Also

Concepts

Connect With Applications (SQL Server PDW)
Client Tools and Applications (SQL Server PDW)


Connect With SQL Server Analysis Services (SQL Server PDW)

[image]

Connect With SQL Server Analysis Services (SQL Server PDW)
See Also Send Feedback

This topic describes how to create a SQL Server PDW data source for Analysis Services, and then provides remarks and recommendations about how to use Analysis Services with SQL Server PDW data. Use SQL Server PDW as a highly performant relational data source for building multidimensional or tabular models with Analysis Services. For example:

  • Use DirectQuery to perform real-time queries against a SQL Server PDW data source from a tabular model.

  • Reduce ROLAP query times by using clustered columnstore indexes on SQL Server PDW tables.

Contents

  • Create a SQL Server PDW Data Source for:

    • SQL Server 2008 R2 Analysis Services

    • SQL Server 2012 Analysis Services - Multidimensional

    • SQL Server 2012 Analysis Services - Tabular

  • General Remarks

  • Limitations and Restrictions

Before You Begin

Before you connect to SQL Server PDW by using Analysis Services, verify that you have met the following requirements.

Software Prerequisites – SQL Server 2008 R2

Use these software prerequisites if you plan to use SQL Server PDW as a data source for SQL Server 2008 R2 Analysis Services.

  • SQL Server 2008 R2 Analysis Services

  • Business Intelligence Development Studio (BIDS). See Introducing Business Intelligence Development Studio.

  • .NET Framework 3.5 SP1 or higher. For installation details, see Microsoft .NET Framework 3.5 Service Pack 1. If you have Windows 7 or later, you already have this.

  • SQL Server Native Client (SNAC) 10.0. Version 10.5, which was included as part of the SQL Server 2008 release, is also supported. For installation instructions, see Install SQL Server Native Client (SQL Server PDW).

Software Prerequisites – SQL Server 2012

Use these software prerequisites if you plan to use SQL Server PDW as a data source for SQL Server 2012 Analysis Services.

  • SQL Server 2012 Analysis Services

  • SQL Server Data Tools. To install the required tools, install the Client Connectivity Tools in your SQL Server 2012 installation. You already have this installed unless you unchecked Client Connectivity Tools in your SQL Server 2012 installation. For more information, see Creating Multidimensional Models using SQL Server Data Tools.

  • .NET Framework 3.5 SP1 or higher. See Microsoft .NET Framework 3.5 Service Pack 1. If you have Windows 7 or later, you already have this.

  • SQL Server Native Client (SNAC) 11.0. See Install SQL Server Native Client (SQL Server PDW).

Create a SQL Server PDW Data Source for SQL Server 2008 R2 Analysis Services

These steps explain how to create a data source that connects to SQL Server PDW. The connection uses SQL Server Native Client 10.0 and connects in the same manner that you would connect to SQL Server.

  1. Open Business Intelligence Development Studio, and create a new project.

  2. In the New Project window, for Project type, select Business Intelligence Projects, and then select the template, Analysis Services Project. Click OK.

  3. In Solution Explorer, under your new project, right-click Data Sources, and select New Data Source.

    Analysis Services 2008 R2 New Project

  4. Click Next to connect.

  5. In the Welcome to the Data Source Wizard screen, click Next.

  6. In the Select how to define the connection screen, choose Create a data source based on an existing or new connection, and click New.

  7. In the Connection Manager window, make the following selections:

    • Provider: Select Native OLE DB\SQL Server Native Client 10.0.

    • For InfiniBand connections, Server name: Enter <appliance-name>-SQLCTL01,17001. For example, PDW001-SQLCTL01,17001.

      For Ethernet connections, Server name: Enter the IP address of the Control node cluster, comma, port 17001. For example, 10.192.63.134,17001.

    • Choose Use Windows Authentication, or choose Use SQL Server Authentication.

    • User name and Password fields: Type your SQL Server PDW login and password if using SQL Server Authentication .

    • Select or enter a Database Name: Select or enter the name of the target database.

      Analysis Services Connection

  8. Click Test Connection to verify that the new data source is available.

  9. Click OK to return to the page, Select how to define the connection.

  10. In the Select how to define the connection window, view your new connection, listed under Data Connections.

  11. Click Finish to bypass the Impersonation Information page.

  12. On the Completing the Wizard page, type a friendly name for your data source.

  13. You can now see your data source listed in Solution Explorer under your Analysis Services Project.

    Data source visible in BIDS Solution Explorer

[ TOP]

Create a SQL Server PDW Data Source for SQL Server 2012 Analysis Services – Multidimensional

  1. Open SQL Server Data Tools, and create a new project.

  2. In the New Project window, for Project type, select Business Intelligence Projects, and then select the template, Analysis Services Multidimensional and Data Mining Project. Click OK.

  3. In Solution Explorer, under your new project, right-click Data Sources, and select New Data Source.

  4. On the Welcome to the Data Source Wizard page of the Data source wizard, click Next.

  5. On the Select how to define the connection page, choose Create a data source based on an existing or new connection, and click New.

  6. In the Connection Manager dialog box, make the following selections:

    • Provider: Select Native OLE DB\SQL Server Native Client 11.0.

    • For InfiniBand connections, Server name: Enter <appliance-name>-SQLCTL01,17001. For example, PDW001-SQLCTL01,17001.

      For Ethernet connections, Server name: Enter the IP address of the Control node cluster, comma, port 17001. For example, 10.192.63.134,17001.

    • Choose Use Windows Authentication, or choose Use SQL Server Authentication.

    • User name and Password fields: Type your SQL Server PDW login and password if using SQL Server Authentication.

    • Select or enter a Database Name: Select or enter the name of the target database.

  7. Click Test Connection to verify that the new data source is available.

  8. Click OK to return to the page, Select how to define the connection.

  9. On the Select how to define the connection page, view your new connection, listed under Data Connections.

  10. Click Finish to bypass the Impersonation Information page.

  11. On the Completing the Wizard page, type a friendly name for your data source.

  12. You can now see your data source listed in Solution Explorer under your Analysis Services project.

[ TOP]

Create a SQL Server PDW Data Source for SQL Server 2012 Analysis Services - Tabular

  1. Open SQL Server Data Tools, and create a new project.

  2. In the New Project window, for Project type, select Business Intelligence Projects, and then select the template, Analysis Services Tabular Project. Click OK.

  3. Wait a few minutes as Analysis Services creates the model, and initializes all properties. At this point the model will contain no data.

    initialized tabular model with no data

  4. On the Model menu, select Import from Data Source.

    Note: If the Model menu item is not visible, click one of the tabular model icons in Solution Explorer.

  5. In the first page of the Table Import Wizard, Connect to a Data Source, select Microsoft SQL Server, and then click Next.

    Caution note Caution

    Note: The data source, Microsoft SQL Server Parallel Data Warehouse, can be used only for connections using the MPP driver. This connection type is not supported from SQL Server 2012.

    choose the SQL Server connection for PDW 2

  6. On the page, Connect to a Microsoft SQL Server Database, make the following selections:

    • Friendly connection name: Type a name that will help you identify the connection.

    • Server name: Type the IP address of the Control Node cluster.

    • Choose Use Windows Authentication, or choose Use SQL Server Authentication.

    • User name and Password fields: Type your SQL Server PDW login and password if using SQL Server Authentication.

      Note: Do not select the option, Save my password. The option to persist security information must be set to FALSE.

    • Database name: Type the name of the database to use, or select a database from the list of databases hosted in PDW 2.

    set connection properties for PDW

  7. Click Next.

  8. On the Impersonation Information page, select Service Account, and click Next.

  9. On the page, Choose How to Import the Data, select one of the following options:

    • Select from a list of tables and views to import: Select tables and/or views to use in your tabular model.

      Note: When using a PDW data source, you should be aware of potential memory limitations in the Analysis Services model and plan your data import accordingly. For more information, see the Remarks section of this topic.

    • Write a query that will specify the data to import: Paste in a query, or click Design to use the Query Builder to select tables and views, create queries, and apply filters.

      For example, the following diagram shows the filtering and data selection options in the query builder. To optimize performance and memory usage in Analysis Services tabular models, you should deselect columns that are not necessary, or create row-based filters.

    you can filter data before importing

    Note that the query builder does not support automatic relationship detection; therefore, there is no option to add related tables, nor can the model designer in SQL Server Data Tools automatically create relationships after the data has been imported to the model. If you know that related tables will be needed in the Analysis Services model, you must add them individually and create the relationships manually using the model designer.

  10. On the page, Specify a SQL Query, verify that the list of tables or the query against the PDW data source is correct, and click Finish.

    If you create a query, be sure to provide a friendly name for the query, because the Import Wizard will use this name as the table name for imported data. You can rename the query later; however, renaming and other metadata changes require that the data be refreshed.

  11. Click Finish.

    The Table Import Wizard will immediately begin to load the specified data into the tabular model. You can reopen the connection and add more tables or queries, or rename the tables in the model.

    status of data import

[ TOP]

General Remarks - Managing Data Size in Analysis Services Applications

When you use SQL Server PDW as a data source for tabular models, consider the following techniques for managing relational data access.

Multidimensional solutions: In SQL Server 2008 R2, you can use SQL Server PDW as a data source for building multidimensional solutions, such as cubes and data mining models. When you create a SQL Server PDW data source for a multidimensional model, the definition of the data source is saved together with other solution metadata, but the data is not actually read until you process the model. At that time, a copy of the data is read into memory, summarized or otherwise processed according to the definition of your model, and the resulting aggregates or values are saved to disk or cache as part of the Analysis Services solution.

Tabular models: When you create a SQL Server PDW data source for a tabular model, data is also loaded into memory and this data is used while defining the model. Typically, to make the design experience better, you would use only a subset of the data. However, when you deploy the model, you must read the entire data set into memory and process the data according to the definition of the model. If it is unlikely that the data will fit into memory, you have several options:

  • You can define partitions, process them, and add more partitions.

  • You can use a test server which contains a subset of records as a data source, and then change the connection string when you deploy the tabular model against a production data source.

  • You can modify your tabular model to use DirectQuery. A tabular model running in DirectQuery mode does not cache data, other than the data initially used to design the mode. Instead, queries against the model are converted from DAX to SQL statements and any calculations required by the model are performed against the relational data source on the fly using the xVelocity in-memory engine.

    In general, DirectQuery requires fewer machine resources compared with ROLAP, because data is not cached in memory. The conversion of DAX to SQL has also been optimized, so simple queries on DirectQuery models might perform better than similar queries on ROLAP models. However, use of columnstore indexes is recommended for optimum performance in DirectQuery mode.

PowerPivot models: Similar considerations apply when you use SQL Server PDW as a data source for a PowerPivot model. However, in PowerPivot you cannot define partitions, and it is expected that you will load the entire set of data into the PowerPivot workbook. Moreover, if you publish the workbook to a SharePoint site as part of a PowerPivot gallery, restrictions may apply on the size of the workbook. Therefore, you may wish to work with a subset of data from SQL Server PDW while building a PowerPivot model.

Note that, when you build a tabular model or a PowerPivot model, you do not create a set of metadata that is included with solution objects, but rather specify the data you want to use through a wizard. Therefore, if you later edit the data source data definition later to use different data, each time you make any change, however small, the previous cache of data is cleared and new data is loaded into memory. This will usually also force all calculations in your workbook or model to be recalculated, with corresponding impacts on performance. For more information about options for managing recalculation, see Manually Recalculate Formulas in PowerPivot.

[TOP]

Restrictions

Using SQL Server PDW data for ROLAP and MOLAP multidimensional models (cubes)

  • When using SQL Server PDW as a data source for ROLAP queries, we recommend that you consider the use of columnstore indexes to improve performance. For more information, see Clustered Columnstore Indexes (SQL Server PDW)

  • Auto-cube refresh is not supported in this release. The auto-cube refresh feature depends on notifications of table data changes to trigger refresh of the cube partitions, but these stored procedures are not available in SQL Server PDW. When you set up your Analysis Services multidimensional database, be sure to deselect all notification refresh options (Tracking tables, Client initiated, and Scheduled polling).

  • Materialized views are not supported in this release. As a result, aggregations cannot be created if the cube partition uses the ROLAP storage mode and its source data is stored in SQL Server PDW.

    The reason is that Analysis Services tries to create indexed views to contain aggregations of the partition values. Because SQL Server PDW does not support creation of the WITH SCHEMABINDING syntax of the CREATE VIEW statement, Analysis Services cannot create the views needed and therefore cannot store the aggregations.

  • Proactive caching is supported only if you use the polling mechanisms provided by Analysis Services.

    If you use ROLAP storage mode, you must disable proactive caching. The reasons is that the current implementation of proactive caching for ROLAP data sources relies on traces created by SQL Server and by the automatic creation of materialized views, neither of which are available in this release.

  • Writeback is not supported, either in ROLAP or MOLAP mode.

Using SQL Server PDW data in a tabular model in DirectQuery mode

  • DAX query conversion: When you use SQL Server PDW as a DirectQuery data source, DAX queries generated by the tabular model are converted into equivalent statements in Transact-SQL and then sent to SQL Server PDW. In some cases the results of those queries might be different than the result you get when you execute the same formula against the data stored in the tabular model cache. The reason is that the xVelocity in-memory data storage engine handles some data types and calculations differently than does Transact-SQL. For a detailed description of the possible semantic differences, see Formula Compatibility in DirectQuery Mode.

    If you are interested in seeing how the DAX formula for a query is converted into Transact-SQL, you can create a trace on the Analysis Services server that hosts the tabular model, and review the DirectQuery events in SQL Profiler. This class of events is not selected by default when you start a trace session using the default template; therefore, be sure to click the Events Selection tab in the Trace Properties dialog box, select the option, Show all events, and then select the DirectQuery events.

  • Data type compatibility and unsupported data types: To enable loading of data from SQL Server to SQL Server PDW, some data types have had to be converted. In other words, not all SQL Server data types are supported. Unsupported types include geometry, xml, and nvarchar(max).

    Generally, the unsupported data types are converted to a compatible type that is supported by SQL Server PDW. For example, xml data might be converted to varchar.

    Conversion to SQL Server PDW data types can lead to overflow errors or truncation of values. For example, the conversion of a datetime data type to a smalldatetime data type results in an out-of-range value.

    If your data contains a data type not supported in PDW and no compatible data type can be found, the data is loaded but the values are nullified. This can create problems in the model.

[TOP]

See Also

Other Resources

Client Tools (SQL Server PDW)


Connect With SQL Server Linked Server (SQL Server PDW)

[image]

Connect With SQL Server Linked Server (SQL Server PDW)
See Also Send Feedback

Describes how to register and use a linked server connection from SQL Server to SQL Server PDW. This enables you to use SQL Server to submit SQL statements to SQL Server PDW by using existing SQL Server linked server technology.

Contents

  • Before You Begin

  • Limitations and Restrictions

  • Security

  • Examples

Before You Begin

Software Prerequisites

SNAC 10.0 for SQL Server 2008 R2. SNAC 11.0 for SQL Server 2012. See Install SQL Server Native Client (SQL Server PDW)

The server that will be configured to make a linked server connection to SQL Server PDW must be running SQL Server Enterprise or Data Center Edition. For example the lazy schema validation option, which we recommend setting to TRUE, is not available in SQL Server Standard Edition.

Limitations and Restrictions

SQL Server PDW cannot be configured to make an outgoing linked server connection.

SQL statements must be submitted using the linked server EXECUTE statement. Using the EXECUTE statement avoids using four-part names for objects, which is not supported by SQL Server PDW. For example:

  • Use this: EXEC ('SELECT * FROM AdventureWorksPDW2012.dbo.dimCustomer') AT PDWLINK;

  • Don’t use this: SELECT * FROM PDWLINK.AdventureWorksPDW2012.dbo.dimCustomer

Other linked server functionality is not supported. For more information about using linked servers see Linking Servers on MSDN.

The linked server provider must be run using the AllowInProcess option. The AllowInProcess option can be set in Management Studio by using the Properties dialog box for the Microsoft.SQLServerMPPOLEDBProvider provider.

To select this, in SQL Server Management Studio, in Object Explorer, expand Linked Servers, expand Providers, right click on SQLNCLI10 and choose Properties. In the Provider Options window, under Provider Options, enable Allow inprocess, and click OK.

Provider options for Linked Server

[TOP]

Security

One way to create a linked server script is to use the linked server UI in SQL Server Management Studio to generate the script. When using this approach, if you enter the user id and password as part of the connection string on the General properties page, the resulting script will contain the user id and password in plain text. To block the plain text password from appearing in the script, enter the user id and password on the Security properties page of the linked server UI.

The following diagram shows the Security page for the linked server UI in SQL Server Management Studio where you should enter your credentials.

Security Properties for Linked Server Page

[TOP]

Examples

Create a Linked Server by Using SQL Server Management Studio

Open SQL Server Management Studio.

  1. Open SQL Server Management Studio.

  2. In Object Explorer, expand Server Objects, right-click on Linked Servers and choose New Linked Server.

  3. On the New Linked Server page, enter the following field information:

    • Linked Server: Enter a name for the Linked Server object.

    • Server Type: Choose other data source.

    • Provider: Select SQL Server Native Client 10.0.

    • Product name: Enter PDW.

    • Provider string: For the Server use the IP address of the Control node cluster, followed by a comma, followed by port 17001. For the Database, enter master or your own database. Add the Pooling=False parameter.

      Server=<control_node cluster_IP>,17001;

      Database=master;

      Pooling=False;

    • Catalog: Same as the Database parameter.

    General Properties for Linked Server

Enter information into the Security Page.

  1. Under the Select a page pane, select Security.

  2. On the Security page, enter the following information.

  3. Login not defined in the list above, connecitons will: choose Be made using this security context.

  4. Remote Login: Enter the login name that will log into SQL Server PDW.

  5. With password: Enter the password for the connection. By using the remote login,

Security Options for Linked Server

Enter information into the Server Options Page.

  1. Under the Select a page pane, select Server Options.

  2. On the Server Options, enter the following information.

  3. RPC: True

  4. RPC Out: True

  5. Collation Compatible: False

  6. Data Access: True

  7. Use Remote Collation: True

  8. Collation name: leave blank

  9. Connection Timeout: 0

  10. Query Timeout: 0

  11. Distributor: False

  12. Publisher: False

  13. Subscriber: False

  14. Lazy Schema Validation: True

  15. Enable Promotion of Distributed Transaction: False

You are ready to create the Linked Server. You can either

Click OK at the bottom right to generate the Linked Server.

Server options for Linked Server

A. Use a Script That Creates a Linked Server From SQL Server to SQL Server PDW

To build a linked server in SQL Server that points to SQL Server PDW, run the script below after editing the connection string to point to your own SQL Server PDW.

Copy image Copy Code
EXEC master.dbo.sp_MSset_oledb_prop N'SQLNCLI', N'AllowInProcess', 1
GO

EXEC master.dbo.sp_addlinkedserver
@server = N'PDWLINK', @srvproduct=N'PDW',
@provider=N'SQLNCLI', @provstr=N'Server=<ip_control_node_cluster>,17001', @catalog=N'<database>';
GO

EXEC master.dbo.sp_addlinkedsrvlogin
@rmtsrvname=N'PDWLINK',@useself=N'False',
@rmtuser=N'<PDW_login_name>',@rmtpassword='<PDW_login_password>'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'rpc',
@optvalue=N'true'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'rpc out',
@optvalue=N'true'
GO

-- The options below are not required
EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'collation compatible',
@optvalue=N'false'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'data access',
@optvalue=N'true'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'dist',
@optvalue=N'false'
GO
EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'pub',
@optvalue=N'false'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'sub',
@optvalue=N'false'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'connect timeout',
@optvalue=N'0'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'collation name',
@optvalue=null
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'lazy schema validation',
@optvalue=N'true'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'query timeout',
@optvalue=N'0'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'use remote collation',
@optvalue=N'true'
GO

EXEC master.dbo.sp_serveroption
@server=N'PDWLINK', @optname=N'remote proc transaction promotion',
@optvalue=N'false'
GO

B. Use EXEC to Submit a Query to a Linked Server

The following is an example of a query submitted to a linked server named PDWLINK by using the EXEC command. The EXEC command is run in SQL Server and the results are obtained from SQL Server PDW.

This example uses EXEC … AT.

Copy image Copy Code
EXEC ('SELECT SUM(SalesAmount) AS Amount, COUNT (DISTINCT ProductKey) AS NumberOfRows
FROM AdventureWorksPDW2012.dbo.FactInternetSales') AT PDWLINK;

This example uses OPENQUERY.

Copy image Copy Code
SELECT * FROM OPENQUERY( PDWLINK, 'SELECT SUM(SalesAmount), COUNT (DISTINCT ProductKey)
FROM AdventureWorksPDW2012.dbo.FactInternetSales');

C. Use Escape Characters To Specify Text in a Linked Server Query

This example uses the single quote (') escape character around single-quoted text in a linked server query. This is required because the entire query is enclosed with single quotes.

Copy image Copy Code
EXEC ('SELECT f.OrderDateKey, f.SalesOrderNumber
FROM
    AdventureWorksPDW2012
  .dbo.FactInternetSales AS f
WHERE f.OrderDateKey > 20040000
AND f.SalesOrderNumber LIKE ''SO67%'';') AT PDWLINK

See Also

Other Resources

Client Tools (SQL Server PDW)
Using Linked Server With SQL Server


Connect With SQL Server Report Builder (SQL Server PDW)

[image]

Connect With SQL Server Report Builder (SQL Server PDW)
See Also Send Feedback

This topic describes how to connect to SQL Server PDW with SQL Server 2008 R2 Report Builder 3.0 and SQL Server 2014 Report Builder.

Contents

  • Before You Begin

  • Create a Data Source (SQL Server 2008 R2 Report Builder 3.0)

  • Create a Data Source (SQL Server 2012 Report Builder)

Before You Begin

Software Prerequisites for SQL Server 2008 R2

  1. SQL Server 2008 R2 Report Builder 3.0. You can use the Report Builder 3.0 standalone version or the ClickOnce version of Report Builder 3.0 installed with Reporting Services. To install the standalone package, see Download Microsoft SQL Server 2008 R2 Report Builder 3.0 on the Microsoft Download Center. For more information, see Installing, Uninstalling and Supporting Report Builder

  2. .NET Framework 3.5 SP1 or higher. If you have Windows 7 or later, you already have this.

Software Prerequisites for SQL Server 2012

  1. SQL Server 2014 Report Builder. You can use the Report Builder standalone version or the ClickOnce version of Report Builder installed with Reporting Services. To install the standalone package, see Download Microsoft SQL Server 2012 Report Builder on the Microsoft Download Center. For more information, see Installing, Uninstalling and Supporting Report Builder

  2. .NET Framework 3.5 SP1 or higher. If you have Windows 7 or later, you already have this.

[TOP]

Create a SQL Server Data Source (SQL Server 2008 R2 Report Builder 3.0)

Create a Report Builder project.

To create an Report Builder 3.0 connection

  1. Open Report Builder. To do this, in Windows click Start, choose All Programs, choose SQL Server 2008 R2 Report Builder 3.0, and then select Report Builder 3.0.

  2. Add a data source. To do this, in the Report Data menu, select New, and choose Data Source….

  3. In the Data Source Properties window, choose General.

    Report Builder 3.0 Data Source Properties

  4. Fill in the General properties with the following selections:

    • Name: Provide a name for this data source.

    • Choose Use a connection embedded in my report.

    • Select connection type: Choose Microsoft SQL Server.

    • To create the Connection String, click Build….

  5. In the Connection Properties window, make the following selections:

    • Data source: Microsoft SQL Server (SqlClient)

    • Server name: Enter the IP address of the Control node cluster, followed by a comma, followed by the port name of 17001. For example 10.192.63.147,17001.

    • Choose Use Windows Authentication, or choose Use SQL Server Authentication.

    • User name and Password fields: enter your SQL Server PDW login and password if using SQL Server Authentication.

    • Select or enter a Database Name: Select or enter the name of the target database.

      Report Builder 3.0 Connection Properties

  6. Click Test Connection to verify that the new data source is available.

  7. Click OK.

  8. You can now view your connection string in the Data Source Properties windows.

    Report Builder 3.0 Connection Properties

  9. Click OK to return to the Report Builder window.

  10. In the Report Data pane, view your new connection listed under Data Sources.

Create a SQL Server Data Source (SQL Server 2012 Report Builder)

Create a Report Builder project.

To create an Report Builder connection

  1. Open Report Builder standalone. To do this, in Windows click Start, click All Programs, click SQL Server 2012 Report Builder, and then click Report Builder.

  2. Add a data source. To do this, in the Report Data pane, select New, and choose Data Source….

  3. In the Data Source Properties window, choose General.

    Report Builder 3.0 Data Source Properties

  4. Fill in the General properties with the following selections:

    • Name: Provide a name for this data source.

    • Choose Use a connection embedded in my report.

    • Select connection type: Choose Microsoft SQL Server.

    • To create the Connection String, click Build….

  5. In the Connection Properties window, make the following selections:

    • Data source: Microsoft SQL Server (SqlClient)

    • Server name: Enter the IP address of the Control node cluster, followed by a comma, followed by the port name of 17001. For example 10.192.63.147,17001.

    • Choose Use Windows Authentication, or choose Use SQL Server Authentication.

    • User name and Password fields: enter your SQL Server PDW login and password if using SQL Server Authentication.

    • Select or enter a Database Name: Select or enter the name of the target database.

      Report Builder 3.0 Connection Properties

  6. Click Test Connection to verify that the new data source is available.

  7. Click OK.

  8. You can now view your connection string in the Data Source Properties windows.

    Report Builder 3.0 Connection Properties

  9. Click OK to return to the Report Builder window.

  10. In the Report Data pane, view your new connection listed under Data Sources.

See Also

Other Resources

Client Tools (SQL Server PDW)


Connect With sqlcmd Command-Line SQL Client (SQL Server PDW)

[image]

Connect With sqlcmd Command-Line SQL Client (SQL Server PDW)
Send Feedback

Describes how to use the SQL Server sqlcmd Command-Line Tool to connect to SQL Server PDW. sqlcmd is a command-line tool that ships with SQL Server; you can also download it directly from MSDN if you don’t have SQL Server installed. Use this tool to send queries to SQL Server PDW.

  • To install sqlcmd, see Install sqlcmd Command-Line Client (SQL Server PDW).

  • For the complete SQL Server documentation, see sqlcmd Utility on MSDN.

sqlcmd Examples

The following example, establishes a connection to the AdventureWorksPDW2012 database. It uses 10.192.63.134 for the Control node Cluster IP address, and port 17001.

Copy image Copy Code
sqlcmd -S "10.192.63.134,17001" -U <login> -P **** -I -d AdventureWorksPDW2012

General Remarks

The –I option sets the SET QUOTED_IDENTIFIER connection option to ON. It is required for SQL Server 2012 sqlcmd connections to SQL Server PDW. We recommend always using –I when connecting from any of the supported versions of SQL Server.

Limitations and Restrictions

Unsupported Options

The following list shows sqlcmd options that do not apply or are not supported for connections to SQL Server PDW.

-A

Specifies to login to SQL Server PDW with a Dedicated Administrator Connection (DAC).

-K application_intent

Specifies to declare the application workload type when connecting to a server.

-C

Specifies to configures the client to implicitly trust the server certificate without validation. This option is equivalent to the ADO.NET option TRUSTSERVERCERTIFICATE = true.

-f codepage | i:codepage[,o:codepage] | o:codepage[,i:codepage]

Specifies the input and output code pages. The codepage number is a numeric value that specifies an installed Windows code page.

-M multisubnet_failover

Specifies to provides faster detection of and connection to the active server of an availability group or failover cluster.

-N

Specifies to use an encrypted connection.


Connect With dwsql Command-Line SQL Client (SQL Server PDW)

[image]

Connect With dwsql Command-Line SQL Client (SQL Server PDW)
See Also Send Feedback

The dwsql command-line SQL client is used to submit SQL statements and script files to SQL Server PDW. dwsql can also be used for sending system commands (such as a call to the Loader utility) to SQL Server PDW.

To install dwsql, install ClientTools-amd64.msi or ClientTools-x86.msi and select dwsql from the installation menu. These two Windows Installer files ship with the appliance. To locate the installation files, ask your administrator for the location. For more information, see Client Tools and Applications (SQL Server PDW).

The latest version connects of dwsql connects by using SQL Server Native Client 10.0 or 11.0.

Important note Important

Deprecation Notice: dwsql is deprecated in this release and will not be supported in future appliance updates. To prepare for this change, we recommend migrating existing processes to use Connect With sqlcmd Command-Line SQL Client (SQL Server PDW) instead of dwsql as soon as possible.

Some of the key features of dwsql are:

  • The ability to both import and export SQL statements from and to local files.

  • Support for conditional logic, including if…then structures and goto <label> options.

  • The ability to store connection and behavioral values as local environment variables.

  • Support for running dwsql commands at the command line in batch files or in an interactive mode.

  • The ability to control behavior of the tool by specifying how errors are handled, how many rows are returned, the column delimiter character, and other settings.

To install dwsql, see Install Integration Services Destination Adapters (SQL Server PDW).

Contents

  • Command Line Arguments

  • Remarks

  • Variable Precedence

  • sqlcmd Variables

  • sqlcmd Commands

  • Scripting Logic

  • dwsql File Names

  • Output Format for Queries

  • Examples

Syntax

dwsql.exe [ [<connection_options>] [<use_options>] ] | [<help_option>]

<connection_options>::
    [ -c connection_string ] |
    [ [ -d database_name ]
    [ -p port_number ]
    [ -P password ]
    [ -S server_or_IP_address ]
    [ -U login_name ] ]

<use_options>::
    [ -C ]
    [ -e ]
    [ -h header_rows ]
    [ -i input_file ] ]
    [ -M ]
    [ -n ]
    [ -o output_file ]
    [ -q query_name [ ,...n ] ]
    [ -Q query_name [ ,...n ] ]
    [ -s column_delimiter ] ]
    [ -v varName = varValue ] [ ,...n ]
    [ -w screen_width ] ]
    [ -x extract_file ]
    [ -Y variable_display ]
    [ -N non_login_encryption_setting ]
    [ -D certificate_name ]

<help_option>::
    [ -? ]

Command-Line Arguments

Connection Options

[ -c connection_string ]

Connection string.

Example

-c "host=10.10.10.10;user id=user1;password=pwd1;port=17000;database=dwsys"

In the connection string, 'port=17000' is the default value and does not need to be specified when connecting to port 17000.

[-d database_name ]

Sets the database context. Issues the USE database_name; statement as the first statement after the connection is made.

[ -p port_number ]

The port number used in the connection string. If not specified, defaults to 17000.

[ -P password ]

The password for use in the connection string. This can also be specified in the DB_LOGON environment variable.

[ -S server_or_IP_address ]

The name of the server to connect to. The name can be the server name or the IP address.

[ -U login_name ]

The login name. This can be specified in the DB_LOGON environment variable.

Use Options

[ -C]

Instructs dwsql to allow and run only sqlcmd commands. Any process flow statements will generate an error.

[ -e]

Instructs dwsql to echo commands before running them.

[ -h header_rows ]

The number of data rows to show between header rows. When positive, this value must be between 1 and the maximum integer value. Alternatively, when header_rows is -,1 header rows are displayed.

[ -i input_file ]

Runs dwsql in batch mode with an input file and exits on completion. If omitted, dwsql starts in interactive mode.

[ -M]

Disables use of the :setvar command so that variables cannot be modified. This is the -x option in sqlcmd.

[ -n]

Instructs dwsql not to require the GO statement to run SQL. A SQL statement will be run once the ';' is detected

[ -o output_file ]

Sends dwsql results to this file.

[ -q query_name [; …]]

Query that will run when the dwsql tool starts. After completion of the query, dwsql will remain open, awaiting additional instructions.

[ -Q query_name [; …]]

Query (or queries) that will run when dwsql tool starts. The dwsql tool will exit when the query (or queries) finish.

[ -s column_delimiter ]

Delimiter for fields when formatting the result set.

[ -v varName = varValue ] [, …n ]

Creates a scripting variable that can be used in dwsql scripts. varValue must be enclosed in quotation marks if it contains spaces. For example, -v MyVar1=something MyVar2="some thing" or -v MyVar1=something -v MyVar2="some thing".

When referencing these variables within dwsql scripts, use this standard variable syntax: $(varName). For example, if the command line included the switch –v TableName=OrdersJan2011, the following INSERT statement would substitute OrdersJan2011 for the variable TableName: INSERT INTO $(TableName) VALUES (1, ‘Denver’, ‘2011-03-11’, ‘OID75426’);

[ -w screen_width ]

Screen width, in number of characters. If any row requires more than this number, it will wrap to the next row, regardless of your actual screen or window size. Default is 100.

[ -x extract_file ]

Specifies a file name to which result sets will be directed. If this file does not already exist, it will be created. If it does exist, any contents will be overwritten.

[ -Y variable_display ]

Limits the display length of the following variable length character types in result sets: char, nchar, varchar, and nvarchar. Max specified size is 8000. If 0 is specified then output is unlimited.

[ -N non_login_encryption_setting ]

Enables data encryption for non-login data. (Login data is always encrypted). Can be "FALSE" (default) or "TRUE".

[ -D certificate_name ]

Validates certificate on the server, based on the certificate name.

Help Option

[ -?]

Shows the command-line arguments.

[TOP]

General Remarks

Options do not have to be used in the order shown in the syntax section.

When multiple results are returned, dwsql prints a blank line between each result set in a batch. In addition, the "<x> rows affected" message does not appear when it does not apply to the statement run.

To use dwsql interactively, type dwsql.exe at the command prompt with any one or more of the options described earlier in this topic.

The total permitted length of the dwsql command line in the command environment (Cmd.exe or PowerShell), including all arguments, is that which is determined by the operating system for Cmd.exe or PowerShell.

TOP

Variable Precedence (Low to High)

  1. System-level environment variables

  2. User-level environment variables

  3. Command shell (SET X=Y) set at the command prompt before running dwsql.

  4. :Setvar X Y

Note Note

To view the environment variables, in Control Panel, open System, and then click the Advanced tab.

TOP

sqlcmd Variables

The following pre-defined variables are supported by dwsql and can be set by using the :setvar command or by using them as environment variables. Some of these variables have corresponding command-line values (switches). See the previous section for usage and descriptions of switches.

Variable | Related Switch | Description | Default | Range
SQLCMDCOLWIDTH | -w | Limits the display length of the following variable-length character types in result sets: char, nchar, varchar, and nvarchar. The default is 256. | 80 | Column width must be a number greater than 8 and less than 65536.
SQLCMDCOLSEP | -s | Delimiter for fields when formatting the result set. | | | ???
SQLCMDCONNECTURL | -c | Specifies a database connection. | NA | host=<serverNameIP>; Port=<PortNum>
SQLCMDDBNAME | -d | Database to use when initially connecting. | NA |
SQLCMDEDITOR | :ed | Specifies a program to start when the :ed command is specified. | Notepad | Notepad, Wordpad
SQLCMDHEADERS | -h | Number of rows shown between headers. | NA | The number of data rows to show between header rows. When positive, this value must be between 1 and the maximum integer value. Alternatively, when header_rows is -1, header rows are displayed.
SQLCMDLINEWIDTH |  | Maximum line width before wrapping. | 100 | 100 up to the maximum int value.
SQLCMDMAXVARTYPEWIDTH | -Y | Limits the display length of the following variable-length character types in result sets: char, nchar, varchar, and nvarchar. Maximum size of varchar / char and others. Max specified size is 8000. If 0 is specified then output is unlimited. | 256 |
SQLCMDNUMBERROWS |  | Maximum number of rows returned. This is functionally equivalent to TOP in a SQL command. | 2000 |
SQLCMDPASSWORD | -P | Current password. Default connection port number. SQLCMDUSER, SQLCMDPASSWORD, SQLCMDSERVER, and SQLCMDCONNECTPORT are set when .connect is used. | NA |
SQLCMDRETURNCODE |  | The status of execution of the dwsql command or SQL queries. Possible values are 0 (Success), 1, and -1. | 0 |
SQLCMDSERVER | -S | Server to connect to, either a server name or an IP address. Default connection port number. SQLCMDUSER, SQLCMDPASSWORD, SQLCMDSERVER, and SQLCMDCONNECTPORT are set when .connect is used. | NA |
SQLCMDUSER | -U | The login account to connect with. Default connection port number. SQLCMDUSER, SQLCMDPASSWORD, SQLCMDSERVER, and SQLCMDCONNECTPORT are set when .connect is used. |  |
SQLCMDCONNECTPORT | -p | Default connection port number. SQLCMDUSER, SQLCMDPASSWORD, SQLCMDSERVER, and SQLCMDCONNECTPORT are set when .connect is used. | 17000 |

TOP

sqlcmd Commands

In addition to SQL statements within dwsql, the following commands are also available.

Be aware of the following when you use these commands:

  • Each command must appear at the start of a line.

  • Each command must be on a separate line. A command cannot be followed by a SQL statement or another command.

  • Commands are run immediately. They are not put in the execution buffer as SQL statements are.

Command

Description

Connection Commands

.connect 'Host=server_name;User id=user_name;Password=password;Port=portNum' [ connectionName ]

Connects to an instance of SQL Server PDW. Also closes the previously active connection, if any.

The SQLCMDSERVER scripting variable will reflect the current active connection.

All parameters must be provided except connectionName.

.session-rename new_connection_name

Renames the current connection to the specified new_connection_name.

.session-switch connection_name

Switches to a connection that was connected to and named during the current session. connection_name is the previously named connection.

set-property

Changes the values for the following session variables:

Property Name | Description | Default Value | Range
Column Delimiter | The character that separates column values in result sets. | Single Space |
echo command | Whether the input command is echoed to the screen before result sets are shown. | off | off, on
sql-result-limit | The maximum number of rows that can be returned to the user. This has the same effect as a TOP ( n ) in a SQL query. | 2000 | 1 - 9,223,372,036,854,775,807, or specify 0 to permit the maximum number of returned rows.
sql-result-showfooter | Whether to show horizontal lines that act as a separator at the end of the query result set. | on | on, off
sql-result-showheader | Whether to show the result set header, which consists of the column names in the query. | on | on, off

Editing Commands

:ed

Starts the text editor. This editor can be used to edit the current batch or the last run batch. To edit the last run batch, the ed command must be typed immediately after the last batch has completed execution.

The text editor is defined by the SQLCMDEDITOR environment variable. The default editor is 'Edit'. To change the editor, set the SQLCMDEDITOR environment variable. For example, to set the editor to Microsoft Notepad at the command prompt, type:

SET SQLCMDEDITOR=notepad

To save edits made within the text editor, use File > Save, or other Save commands within the tool to save without changing the file name or location.

:reset

Clears the statement cache.

:list

Prints the content of the statement cache.

Variable Management

:listvar

Displays a list of the scripting variables that are currently set.

Note Note

Only scripting variables that are set by using dwsql or the :Setvar command will be displayed.

:setvar var [ "value" ]

Defines dwsql scripting variables. Scripting variables have the following format: $(VARNAME).

Variable names are not case-sensitive.

Scripting variables can be set in the following ways:

  • Implicitly, using a command-line option. For example, the -U option sets the SQLCMDUSER dwsql variable.

  • Explicitly by using the :Setvar command.

  • By defining an environment variable before you run dwsql.

If a variable defined by using :Setvar and an environment variable have the same name, the variable defined by using :Setvar takes precedence.

Variable names must not contain blank space characters.

Variable names cannot have the same form as a variable expression, such as $(var).

If the string value of the scripting variable contains blank spaces, enclose the value in quotation marks. If a value for a scripting variable is not specified, the scripting variable is dropped.

Output Commands

:error filename | STDERR | STDOUT

Redirect all error output to the file (specified by file name) to stderr or to stdout. The Error command can appear multiple times in a script. By default, error output is sent to stderr.

filename

Creates and opens a file that will receive the output. If the file already exists, it will be truncated to zero bytes. If the file is not available because of permissions or other reasons, the output will not be switched and will be sent to the last specified or default destination.

STDERR

Switches error output to the stderr stream. If this has been redirected, the target to which the stream has been redirected will receive the error output.

STDOUT

Switches error output to the stdout stream. If this has been redirected, the target to which the stream has been redirected will receive the error output.

:out filename | STDERR| STDOUT

Creates and redirects all query results to the file specified by filename to stderr or to stdout. By default, output is sent to stdout. If the file already exists, it will be truncated to zero bytes. The Out command can appear multiple times in a script.

Execution Control Commands

:quit

Causes dwsql to exit.

GO

GO signals both the end of a batch and the execution of any cached SQL statements.

:on error[ exit | ignore ]

Sets the action to be performed when an error occurs during script or batch execution.

When the exitoption is used, dwsql exits with the appropriate error value.

When the ignore option is used, dwsql ignores the error and continues running the batch or script.

By default, an error message will be printed.

Miscellaneous Commands

:r filename

Parses additional SQL statements and dwsql commands from the file specified by filename into the statement cache.

If the file contains SQL statements that are not followed by GO, you must enter GO on the line that follows :r.

Note Note

filename is read relative to the startup directory in which dwsql was run.

The file will be read and run after a batch terminator is encountered. You can issue multiple :r commands. The file may include any dwsql command. This includes the batch terminator GO.

Note Note

The line count that is displayed in interactive mode will be increased by one for every :r command encountered. The :r command will appear in the output of the list command.

:!! Command

Runs operating system commands. To run an operating system command, start a line with two exclamation marks (!!) followed by the operating system command. For example:

:!! Dir

Note Note

The command is run on the computer on which dwsql is running.

TOP

Scripting Logic

Within dwsql scripting files and ad hoc requests, the following statements can be used to create loops, compare values, and perform other scripting functions.

Statement Name

Description

Commenting Statements

# comment_line

Creates one line of commented text.

/* comment_block */

Creates a comment block that can include multiple lines.

-- comment_line

Creates one line of commented text.

Conditional Logic and Value Management

.compare value1 value2

Compares two query return values. Returns 1 if the values do not match. Returns 0 if the values do match. Typically, .compare would be used to compare results previously saved with .saveresults.

.goto label_name

Directs the script execution to proceed to either a label or "exit". See .label below.

.if test_value comparator action

A conditional if statement, where:

test_value is "retcode" (the current return code), "rowcount" (the row count returned by the last SQL operation), or a field previously saved with .saveresults.

comparator is one of the following: "eq" (equal to), "ne" (not equal to), "gt" (greater than), "ge" (greater than or equal to), "lt" (less than), or "le" (less than or equal to).

action is either "goto <label_name>" or "quit". When "goto <label_name>", the label must be specified elsewhere in the file or sequence. See .label below.

.label label_name

A label that will be referenced by an if statement, where

label_name is a string value without spaces. For example, .label NoError.

.replace value1 value2

Replaces the string specified as value1 with the value in value2 in any subsequent SQL statements. If value2 is in the format '$()', it will replace the field with the information from the environment variable. If field2 is the result of a previous .saveresults command, it will replace the value with the information returned from the SQL operation. Once a .replace function begins, it will continue until it has processed all rows or values in the request.

.saveresults AS results_name

Saves the results of the following SQL operation with the results name specified. The results name is not a file name, but is a memory variable that holds the results for future reference.

.set retcode return_code

Changes the value of the return code to the specified value.

Error Handling

.maxerrors [ on | off ]

Specifies whether the script should end on any non-zero return code. Default is 'off'.

External File Management

.echo [ output | extract ] [ on | off ]

Turns echoing of output or extract information on or off. This command can be used at any point in a script.

.extract [ OPEN file_identifier file_name ] |

[ CLOSE pfile_identifier ]

Allows creation of extract files with the information returned back from SQL operations. Any extract file that is currently open is written to so if you have multiple extract files open, each will receive results. For each row returned, there will be an output line with the columns separated by the column delimiter character.

Value | Description
OPEN | Opens a file to receive result sets. The file will be created if it does not exist. It will be wiped and recreated if it already exists.
CLOSE | Closes an extract open file.

See Example C in Examples for usage of the .extract command.

.extract_hdr info

This command can be used with the -x option or with the .extract commands. The text specified in info will be the first text written to every line of results in the .extract / -x extract file. info cannot include spaces.

.file

[ OPEN | APPEND file_identifier file_name ] |

[ WRITE file_identifier options ] |

[ CLOSE file_identifier ]

This command will allow you to write out log files with information about executed SQL operations, such as start time, end time, and so on.

Value

Description

OPEN | APPEND

Opens a file for writing.

With OPEN, the file will be created if it does not exist. It will be wiped and recreated if it already exists.

With APPEND, the file will be created if it does not exist. Any existing information already in the file will be preserved. New information will be written at the end of the file.

WRITE

Writes to a previously opened file.

CLOSE

Closes an open file.

file_identifier

A string name for the file for this session. file_identifier can be any valid string.

file_name

The path and name of the file.

options

The information that will be written to the file.

options can be one or more of the Query Metadata Variables shown below. Use commas to separate multiple values. options can also be an unrecognized value, which will be written to the file as is.

Query Metadata Variables

dwsql includes the following metadata variables that can be used, typically to include metadata when logging.

Variable Name | Description
{SQL_STARTED} | The time the SQL operation was started.
{FIRST_SQL} | The time the first return operation for the current SQL operation was complete.
{NUM_ROWS} | The number of rows returned by the SQL command. When an error is returned, this value is -1. Otherwise, this is: ‘0’ for commands affecting zero rows, including commands such as CREATE TABLE which never affect rows; the positive integer number of rows affected by the INSERT, SELECT, or other command which can affect multiple rows.
{RETURN_CODE} | Current return code. Potential values are 0 (success), -1 (syntax error), or -2 (execution failure).
{SQL_DURATION} | The amount of time (in seconds) that the last SQL statement required to process. If the statement is still processing, this is the amount of time so far. Otherwise, it is the total processing time.
{SQL_ENDED} | The date and time the SQL operation was completed.
{TOT_SQL_STARTED} | The date and time the first SQL operation in a batch was started.
{TOT_FIRST_SQL} | The total time (duration) in seconds for all the FIRST_SQL operations.
{TOT_NUM_ROWS} | Total number of rows returned by all SQL operations. See {NUM_ROWS} above for information on the values returned for each SQL operation.
{TOT_SQL_DURATION} | The amount of time (in seconds) that all SQL operations required to process. If any operations are still processing, this is the amount of time so far. Otherwise, it is the total processing time.
{TOT_SQL_ENDED} | The date and time the last SQL operation was completed.

.include file_name

Allows inclusion of other files in the stream that will be run. When the file name begins with "$(' and Format is like "$(VarName)/<file>, it is assumed to be an environment variable and the value will be substituted. If the file name is not an absolute path (in the form /<dir>/<file>), it will append the information to the current directory.

Example

$(VarName)/file - If the environment variable VarName is set to /home/user, the file that will be read is /home/user/file

test/file - If the current directory is /home/user, the file that will be read is /home/user/test/file

/home/sam/file - The file /home/sam/file will be read

System Commands

.system command

Runs a specified system command. .system is often used to run another ssDW command-line utility, such as the dwloader Command-Line Loader (SQL Server PDW).

TOP

dwsql File Names

dwsql input and output file names should follow these guidelines:

  • :Error, and :Out should use a separate filename. If the same filename is used, inputs from the commands may be intermixed.

  • If an input file located on a remote server is called from dwsql on a local computer and the file contains a drive file path such as :out c:\OutputFile.txt, the output file will be created on the local computer and not on the remote server.

  • Valid file paths include: C:\filename,\\Server\Share$\filename and "C:\foldername\filename". If there is a space in the path, use quotation marks.

  • Each new dwsql session will overwrite existing files that have the same names.

[TOP]

Output Format for Queries

dwsql first prints a column header that contains the column names specified in the select list. The column names are separated by using the SQLCMDCOLSEP character. By default, this is a pipe ('|'). If the column name is shorter than the column width, the output is padded with spaces up to the next column.

This line will be followed by a separator line that is a series of dash characters. The following output shows an example.

Start dwsql. At the dwsql command prompt, type the following:

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
SELECT TOP (2) FirstName, LastName, EmployeeKey
SELECT TOP (2) FirstName, LastName, EmployeeKey
FROM DimEmployee;
GO

When you press ENTER, the following result set is returned.

FirstName LastName EmployeeKey

---------------- ------------ ----------

Guy Gilbert 1

Kevin Brown 2

Although the EmployeeKey column is only one character wide, it has been expanded to accommodate the longer column name. By default, output is terminated at 80 characters. This can be changed by using the -w option or by setting the SQLCMDCOLWIDTH scripting variable.

[TOP]

Examples

A. Loading dwsql without parameters

dwsql can be loaded without any parameters. In this case, the tool loads in interactive mode and provides a prompt for the user to enter connection information, commands, and variables.

To load dwsql without parameters, specify

Copy image Copy Code
dwsql.exe

Once the tool has started, a connection can be established using the .connect command.

Copy image Copy Code
.connect 'Host=10.10.10.10;User id=user1;Password=pwd1;Port=17000'

B. Loading dwsql with connection switches

dwsql can be loaded with connection parameters as switches. In this case, the tool connects to the specified appliance and database. The tool then loads in interactive mode, providing the user with the ability to enter commands, variables, or new connection information via command line.

The following example uses the -c switch to instruct dwsql to connect to the appliance at 10.10.10.10 and database db1.

Copy image Copy Code
dwsql.exe -c "host=10.10.10.10;user id=user1;password=pwd1;port=17000;database=db1"

Alternatively, the same connection could be established using the -d, -P, -S, and -U switches:

Copy image Copy Code
dwsql.exe -S 10.10.10.10 -U user1 -P pwd1 -d db1

C. Running dwsql with an input file

When dwsql is loaded with an input file (specified by the -i command line switch), the tool loads in batch mode. It runs all instructions in the input file and then exits.

The following command loads dwsql in batch mode, with an input file from the local directory:

Copy image Copy Code
dwsql.exe -i SQLBatch.sql

The following shows the contents of the input file. Comments in the input file show which actions are performed.

Copy image Copy Code
# -- SCRIPT:  ~\SQLBatch.sql
# -- Purpose: This script connects to an appliance, retrieves a list of all databases from sys.databases,
#             and saves the list to an external file in the local directory using the .extract command.
# Connect to the appliance
.connect 'Host=10.10.10.10;User id=user1;Password=pwd1;Port=17000'

# Open a file to receive the results of the SQL statement
.extract OPEN dbNamesFile dbNames.txt

# Execute SQL statement
SELECT name FROM sys.databases;
GO

# Close the extract file
.extract CLOSE dbNamesFile

D. Using variables in dwsql

The following command loads dwsql in batch mode, with an input file from the local directory:

Copy image Copy Code
dwsql.exe -i SQLBatch2.sql

The input file uses the .saveresults command to save results from one query for use as input in a later query. Comments in the script show how each command is used.

Copy image Copy Code
# -- SCRIPT:  ~\SQLBatch2.sql
# -- Purpose: This script connects to an appliance, retrieves the current session ID, then retrieves the list of queries
#             for this session. The return values are written to a file.

# Connect to the appliance
.connect 'Host=dwt02ctl1;User id=sa;Password=test;Port=17000'

# Open a file to receive the results of the SQL statement
 .extract OPEN sessionInfoFile sessionInfo.txt

# Retrieves the most recent (current) session ID and saves it under the name 'SESSION_ID'
.saveresults as SESSION_ID
SELECT TOP (1) session_id FROM sys.dm_pdw_exec_sessions WHERE session_id = SESSION_ID() OPTION (LABEL='Get PDW Statistics');
GO

# Execute second query to retrieve all information regarding that session.
SELECT * FROM sys.dm_pdw_exec_requests WHERE session_id = '$(SESSION_ID)';
GO

# close the extract file
.extract CLOSE sessionInfoFile

[TOP]

See Also

Reference

dwloader Command-Line Loader (SQL Server PDW)

Other Resources

Client Tools (SQL Server PDW)


Connect With PowerShell (SQL Server PDW)

[image]

Connect With PowerShell (SQL Server PDW)
Send Feedback

Windows PowerShell exposes the hierarchy of SQL Server objects in paths similar to file system paths. You can use the paths to locate an object, and then use methods to perform actions on the objects. For more information about using SQL Server and PowerShell, see SQL Server PowerShell Provider.

Examples

In all examples, replace xxx.xxx.xxx.xxx with the IP address of your SQL Server PDW. Replace [login] with the login you use to connect to SQL Server PDW. Replace [*********] with the password for the login. Use TCP port 17000 for SQL Server 2008 PDW. Use TCP port 17001 for SQL Server 2012 PDW.

To start PowerShell:

  1. Open a command prompt.

  2. To start PowerShell execute PowerShell.exe.

Example A: Simple Example

The following example:

  1. Connects to SQL Server PDW.

  2. Defines and executes a SQL statement.

  3. Closes the connection

  4. Returns the statement results to the screen.

Copy image Copy Code
$conn = New-Object System.Data.SqlClient.SqlConnection
$ConnectionString = "Server=xxx.xxx.xxx.xxx,17001;Integrated Security=False;user id=[login];password=[*********]"
$conn.ConnectionString=$ConnectionString
$conn.Open()

$Query = "SELECT database_id, name FROM sys.databases;"
$cmd=new-object system.Data.SqlClient.SqlCommand($Query,$conn)
$ds=New-Object system.Data.DataSet
$da=New-Object system.Data.SqlClient.SqlDataAdapter($cmd)

[void]$da.fill($ds)
$conn.Close()

$ds.Tables

Example B: Simple Example with Variables

The following example: connects to SQL Server PDW, creates a connection, defines and executes a SQL statement, closes the connection, and returns the statement results to the screen.

  1. Defines variables.

  2. Connects to SQL Server PDW using some of the variables.

  3. Builds and executes a SQL statement using some of the variables.

  4. Closes the connection

  5. Returns the statement results to the screen.

Copy image Copy Code
$Server="xxx.xxx.xxx.xxx,17001"
$User="[login]"
$Pwd="[*********]"
$Database="AdventureWorksPDW2012"
$Table="DimCustomer"

$conn = New-Object System.Data.SqlClient.SqlConnection
$ConnectionString = "Server=$Server;Integrated Security=False;user id=$User;password=$Pwd"
$conn.ConnectionString=$ConnectionString
$conn.Open()

$Query = "SELECT COUNT(*) AS NumberOfCustomers FROM $Database.dbo.$Table;"
$cmd=new-object system.Data.SqlClient.SqlCommand($Query,$conn)
$ds=New-Object system.Data.DataSet
$da=New-Object system.Data.SqlClient.SqlDataAdapter($cmd)

[void]$da.fill($ds)
$conn.Close()

$ds.Tables

Example C: Executing a SQL Command in a PowerShell Script

Save the following text in a PowerShell script file named TestScript.ps1 in a folder named C:\PDWScripts. The Server, Port, User, and Pwd variables are passed to PowerShell from the command line.

Copy image Copy Code
#Set the input variables
param($Server, $Port, $User, $Pwd)

#Open a connection
$conn = New-Object System.Data.SqlClient.SqlConnection
$ConnectionString = "Server=$Server,$Port;Integrated Security=False;user id=$User;password=$Pwd"
$conn.ConnectionString=$ConnectionString
$conn.Open()

#Create and execute the SQL script
$Query = "SELECT database_id, name FROM sys.databases;"
$cmd=new-object system.Data.SqlClient.SqlCommand($Query,$conn)
$ds=New-Object system.Data.DataSet
$da=New-Object system.Data.SqlClient.SqlDataAdapter($cmd)

#Clean up
[void]$da.fill($ds)
$conn.Close()

#Return results to the screen
$ds.Tables

Execute the following command at a command prompt. The Server, Port, User, and Pwd variables are passed to PowerShell from the command line.

Copy image Copy Code
PowerShell.exe C:\PDWScripts\TestScript.ps1 -Server xxx.xxx.xxx.xxx –Port [n] -User [login] –Pwd [*********]


Connection Strings (SQL Server PDW)

[image]

Connection Strings (SQL Server PDW)
Send Feedback

This lists the connection string reference topics for connecting to SQL Server PDW.

Reference Topics for Connection Strings

Each connection method listed in the table below has specific connection string keywords. The topics listed in the table provide tools requirements and installation instructions, connection string keywords, and code samples where appropriate.

Data Access API | Data Access Driver or Provider | Reference Topic
ADO.NET | .NET Framework Data Provider for SQL Server | Connection Strings for ADO.NET and .NET Framework Data Provider for SQL Server (SQL Server PDW)
OLE DB | SQL Server Native Client 10.0 | Connection Strings for OLE DB and SQL Server Native Client (SQL Server PDW)
ODBC | SQL Server Native Client 10.0 | Connection Strings for ODBC and SQL Server Native Client (SQL Server PDW)
JDBC | Microsoft JDBC Driver 4.0 for SQL Server | Connection Strings for JDBC and Microsoft JDBC Driver 4.0 for SQL Server (SQL Server PDW)


Connection Strings for ADO.NET and .NET Framework Data Provider for SQL Server (SQL Server PDW)

[image]

Connection Strings for ADO.NET and .NET Framework Data Provider for SQL Server (SQL Server PDW)
See Also Send Feedback

This topic contains connection string keywords, and other connection information that application developers need for connecting to SQL Server PDW with ADO.NET by using the .NET Framework Data Provider for SQL Server.

Contents

  • Basics

  • Before You Begin

  • Connection String Keywords

  • Establish a Secure Connection

  • Connection Example

Basics

.NET Framework Data Provider for SQL Server is the Microsoft core data technology for accessing SQL Server with languages targeting the Common Language Runtime (CLR). This managed provider is installed with .NET Framework 3.5 SP1 or higher. To connect with ADO.NET by using .NET Framework Data Provider, use the System.Data.SqlClient namespace, and specify port 17001.

This provider uses the System.Data.SqlClient namespace.

  • Application developers use these connection strings to programmatically connect to SQL Server PDW.

  • Application users specify some of these connection string values, as required by the application, when connecting to SQL Server PDW.

Before You Begin

Software Prerequisites

To connect, you need .NET Framework 3.5 SP1 or higher.

  • If you have Windows 7, you already have .NET Framework 4.0 and don’t need to install anything else.

  • If you are running a version of Windows prior to Windows 7, you need .NET Framework 3.5 SP1 or higher. You might already have it installed. For installation details, see the Microsoft .NET Framework 3.5 Service Pack 1 page on the Microsoft Download Center.

Appliance Privileges

To connect, you will need a login with privileges to connect and run queries on the appliance.

Learn More

For more information about ADO.NET, see the ADO.NET Managed Providers and DataSet home page on MSDN.

For more information about using System.Data.SqlClient, see Using the .NET Framework Data Provider for SQL Server on MSDN. This link points to the 2.0 version of the data provider; this version ships with .NET Framework 3.5 SP1 and higher.

Connection String Keywords

This table lists the supported connection string keywords for connecting to SQL Server PDW. For full descriptions of each string keyword, connection string creation information, and usage of the APIs, see SqlConnection.ConnectionString Property, .NET Framework 3.5 for .NET Framework 3.5 or SqlConnection.ConnectionString Property, .NET Framework 4 for .NET Framework 4.

The following table lists the keywords that can be used with an ADO.NET connection string:

Name | SQL Server PDW Support Notes
Application Name |
Connection Timeout |
Connection Reset |
Current Language | Supports default value only.
Data Source |
Encrypt |
Initial Catalog |
Max Pool Size |
Min Pool Size |
Network Library | ‘TCP’ only.
Packet Size |
Password |
Persist Security Info |
Pooling |
TrustServerCertificate |
Type System Version |
User ID |
Workstation ID |

[TOP]

Establish a Secure Connection

To establish a secure connection to SQL Server PDW via a managed ADO.NET connection, the connection string must contain the following:

  1. The ValidateServerCertificate value set to true.

  2. When the SSL certificate has a dnsName value in the subjectAltName value, the HostNameInCertificate value in the connection string must match the dnsName in the certificate.

  3. When the certificate does not have a dnsName value in the subjectAltName field or no subjectAltName value is present, the HostNameInCertificate value in the connection string must match the commonName (CN) part in the Subject name of the certificate.

    If multiple commonName parts exist in the Subject name of the certificate, the connection succeeds if the HNIC value matches any of the Common Name parts.

[TOP]

Connection Example

To access SQL Server with ADO.NET:

  1. Use the System.Data.SqlClient namespace.

  2. Use the DbProviderFactories class to connect to SQL Server PDW. Note, this class is not exposed in the Solution Explorer of Visual Studio.

  3. To create a .NET connection, add the line using System.Data.Common in the program header. To add the line in Visual Studio, right-click DBConnection, select Resolve, and select using System.Data.Common.

The following sample code creates a connection and performs a query. Replace <hostIP>, <dbname>, <login>, and <password> with appropriate values for your environment.

Copy image Copy Code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.Common;
namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            using (DbConnection connection = DbProviderFactories.GetFactory("System.Data.SqlClient").CreateConnection())
            {
                connection.ConnectionString = "SERVER=<hostIP>,17001;DATABASE=<dbname>;USER ID=<login>;PASSWORD=<password>";
                connection.Open();
                DbCommand command = connection.CreateCommand();
                command.CommandText = "SELECT TOP 10 * FROM account";
                DbDataReader reader = command.ExecuteReader();
                while (reader.Read())
                {
                    System.Console.WriteLine(reader.GetValue(0));
                }
                reader.Close();
                System.Console.ReadLine();
            }
        }
    }
}

[TOP]

See Also

Other Resources

Client Tools (SQL Server PDW)


Connection Strings for OLE DB and SQL Server Native Client (SQL Server PDW)

[image]

Connection Strings for OLE DB and SQL Server Native Client (SQL Server PDW)
Send Feedback

This topic lists the supported connection string keywords used for connecting to SQL Server PDW with OLE DB by using the SQL Server Native Client 10.0 (SNAC).

Developers use these connection strings to programmatically connect to SQL Server PDW. Application users specify some of these connection string keywords, as required by the application, when connecting to SQL Server PDW.

Contents

  • Before You Begin

  • Connection String Keywords

  • Establish a Secure Connection

Before You Begin

Software Requirements

  • SQL Server Native Client 10.0 (SNAC). See Install SQL Server Native Client (SQL Server PDW).

Connection String Keywords

This section lists keywords that can be used with the SQL Server Native Client OLE DB driver to create connection strings for SQL Server PDW. This is a subset of keywords supported when connecting to SQL Server. Also included are additional instructions when keyword behavior with SQL Server PDW differs from SQL Server.

For full descriptions of each string keyword, connection string creation information, and usage of the APIs, see the OLE BC section in Using Connection String Keywords with SQL Server Native Client.

There are two ways OLE DB applications can initialize data source objects for connecting to SQL Server PDW:

  • IDBInitialize::Initialize

  • IDataInitialize::GetDataSource

The following table describes the keywords that may be used with IDBInitialize::Initialize when connecting to SQL Server PDW.

Keywords for IDBInitialize::Initialize

The following table describes the keywords that may be used with IDBInitialize::Initialize when connecting to SQL Server PDW.

Keyword | Initialization property | SQL Server PDW Support Notes
Addr | SSPROP_INIT_NETWORKADDRESS |
Address | SSPROP_INIT_NETWORKADDRESS |
APP | SSPROP_INIT_APPNAME |
AttachDBFileName | SSPROP_INIT_FILENAME |
Auto Translate | SSPROP_INIT_AUTOTRANSLATE |
AutoTranslate | SSPROP_INIT_AUTOTRANSLATE |
Database | DBPROP_INIT_CATALOG |
DataTypeCompatibility | SSPROP_INIT_DATATYPECOMPATIBILITY |
Encrypt | SSPROP_INIT_ENCRYPT |
Language | SSPROPT_INIT_CURRENTLANGUAGE | ‘English’ only.
MarsConn | SSPROP_INIT_MARSCONNECTION | ‘no’ only.
Net | SSPROP_INIT_NETWORKLIBRARY |
Network | SSPROP_INIT_NETWORKLIBRARY | ‘dbmssocn’ only (TCP)
Network Library | SSPROP_INIT_NETWORKLIBRARY |
PacketSize | SSPROP_INIT_PACKETSIZE |
PersistSensitive | DBPROP_AUTH_PERSIST_SENSITIVE_AUTHINFO |
PWD | DBPROP_AUTH_PASSWORD |
Timeout | DBPROP_INIT_TIMEOUT |
Trusted_Connection | DBPROP_AUTH_INTEGRATED | ‘no’ only.
TrustServerCertificate | SSPROP_INIT_TRUST_SERVER_CERTIFICATE |
UID | DBPROP_AUTH_USERID |
UseProcForPrepare | SSPROP_INIT_USEPROCFORPREP |
WSID | SSPROP_INIT_WSID |

Keywords for IDataInitialize::GetDataSource

The following table describes the keywords that may be used with IDataInitialize::GetDataSource when connecting to SQL Server PDW:

Keyword | Initialization property | ssDW Usage Notes
Application Name | SSPROP_INIT_APPNAME |
Auto Translate | SSPROP_INIT_AUTOTRANSLATE |
AutoTranslate | SSPROP_INIT_AUTOTRANSLATE |
Connect Timeout | DBPROP_INIT_TIMEOUT |
Current Language | SSPROPT_INIT_CURRENTLANGUAGE | ‘English’ only.
DataTypeCompatibility | SSPROP_INIT_DATATYPECOMPATIBILITY |
Initial Catalog | DBPROP_INIT_CATALOG |
MARS Connection | SSPROP_INIT_MARSCONNECTION | ‘False’ only.
Network Address | SSPROP_INIT_NETWORKADDRESS |
Network Library | SSPROP_INIT_NETWORKLIBRARY |
Packet Size | SSPROP_INIT_PACKETSIZE | ‘0’ only.
Password | DBPROP_AUTH_PASSWORD |
Persist Security Info | DBPROP_AUTH_PERSIST_SENSITIVE_AUTHINFO |
Provider |  |
Trust Server Certificate | SSPROP_INIT_TRUST_SERVER_CERTIFICATE |
Use Encryption for Data | SSPROP_INIT_ENCRYPT |
User ID | DBPROP_AUTH_USERID |
Workstation ID | SSPROP_INIT_WSID |

[TOP]

Establish a Secure Conection

To establish a secure connection to SQL Server PDW via an ODBC or OLE DB connection, the connection string must contain the following:

  1. The ValidateServerCertificate value set to true.

  2. The HostNameInCertificate (HNIC) value is optional. It can be used as an extra validation, and when used, complies with the following HNIC values:

    • When the certificate has a dnsName value in the subjectAltName value, the HostNameInCertificate value in the connection string must match the dnsName

    • When the certificate does not have a dnsName value in the subjectAltName field or no subjectAltName value is present, the HostNameInCertificate value in the connection string must match the commonName (CN) part in the Subject name of the certificate. If multiple commonName parts exist in the Subject name of the certificate, the connection succeeds if the HNIC value matches any of the Common Name parts.

[TOP]


Connection Strings for ODBC and SQL Server Native Client (SQL Server PDW)

[image]

Connection Strings for ODBC and SQL Server Native Client (SQL Server PDW)
Send Feedback

This topic describes connecting to SQL Server PDW with ODBC by using SQL Server Native Client 10.0.

Contents

  • Before You Begin

  • Add SQL Server PDW as an ODBC Data Source

  • Connection String Keywords

  • Establish a Secure Conection

Before You Begin

Software Requirements

  • SQL Server Native Client 10.0. See Install SQL Server Native Client (SQL Server PDW)

Add SQL Server PDW as an ODBC Data Source

Before an application can to connect to SQL Server PDW by using the SQL Server Native Client ODBC driver, you need to add SQL Server PDW as a data source.

The following steps show how to add your SQL Server PDW appliance as a data source by using the ODBC Data Source Administrator that ships with Windows. The dialogs may vary among Windows versions.

  1. Open the ODBC Data Source Administrator in Windows.

    1. On your Windows system, click Start, and enter odbcad32.

  2. In the ODBC Data Source Administrator window:

    1. Click the System DSN tab, and then click Add. This will open the Create New Data Source window.

    2. In the Create New Data Source window, select SQL Server Native Client 10.0, and click Finish.

    Add a data source connection.

  3. In the Create a New Data Source to SQL Server window, enter the following:

    • Name / Description: Enter the human-readable name and description for the data source. Make a note of the Data Source Name. You will need this information later.

    • Server: Enter the IP address of the appliance Control node. Append a comma and then port 17001. For example, 10.192.45.136,17001, and click Next.

      Specify the server IP , follows by 17001

  4. In response to the question How should SQL Server verify the authenticity of the login ID?

    1. Choose With Integrated Windows authentication to use Windows Authentication. Choose With SQL Server authentication using a login name and password entered by the user to use SQL Server Authentication.

    2. Uncheck Connect to SQL Server to obtain default settings for the additional configuration options., and click Next.

      Choose SQL Server Authentication.

  5. On the page with database options and ANSI options, leave everything unchanged, and click Next.

    ODBC Ansi settings.

  6. Leave remaining options unchanged, and click Finish.

    Use SQL Server default settings.

  7. Click OK to skip running the Test Connection.

    Skip testing the data source.

  8. Verify that your new data source is listed in the System Data Sources, and uses the SQL Server Native Client 10.0 driver. Click OK to close the window.

    The new data source has been added.

  9. Your data source is now available for applications that need to connect to your appliance.

Connection String Keywords

This table lists the keywords that can be used for connecting to SQL Server PDW with ODBC and SQL Server Native Client. This is a subset of keywords supported when connecting to SQL Server. Also included are additional instructions when keyword behavior with SQL Server PDW differs from SQL Server. For full descriptions of each string keyword, connection string creation information, and usage of the APIs, see the ODBC section in Using Connection String Keywords with SQL Server Native Client.

Keyword | SQL Server PDW Support Notes
Addr |
Address | Only accepts TCP/IP protocol (default). Port must be 17001.
APP |
AutoTranslate |
Database |
Driver |
DSN |
Encrypt |
Fallback |
FileDSN |
Language | ‘English’ only.
MARS_Connection | ‘no’ only.
Net |
Network | ‘dbmssocn’ only.
PWD |
QueryLog_On |
QueryLogFile |
QueryLogTime |
Regional |
SaveFile |
Server | No named instances. Protocol must be “TCP/IP”. Port is 17001.
StatsLog_On |
StatsLogFile |
Trusted_Connection |
TrustServerCertificate |
UID |
UseProcForPrepare |
WSID |

[TOP]

Establish a Secure Conection

To establish a secure connection to SQL Server PDW via an ODBC or OLE DB connection, the connection string must contain the following:

  1. The ValidateServerCertificate value set to true.

  2. The HostNameInCertificate (HNIC) value is optional. It can be used as an extra validation, and when used, complies with the following HNIC values:

    • When the certificate has a dnsName value in the subjectAltName value, the HostNameInCertificate value in the connection string must match the dnsName

    • When the certificate does not have a dnsName value in the subjectAltName field or no subjectAltName value is present, the HostNameInCertificate value in the connection string must match the commonName (CN) part in the Subject name of the certificate. If multiple commonName parts exist in the Subject name of the certificate, the connection succeeds if the HNIC value matches any of the Common Name parts.

[TOP]


Connection Strings for JDBC and Microsoft JDBC Driver 4.0 for SQL Server (SQL Server PDW)

[image]

Connection Strings for JDBC and Microsoft JDBC Driver 4.0 for SQL Server (SQL Server PDW)
Send Feedback

This topic describes how to connect to SQL Server PDW with JDBC by using Microsoft JDBC Driver 4.0 for SQL Server. You can use JDBC to write Java-based tools and applications that connect to SQL Server PDW.

Contents

  • Before You Begin

  • Connection String Keywords

  • Supported Data Types

Before You Begin

Software Requirements

  • Microsoft JDBC Driver 4.0 for SQL Server

  • Java 1.6 Update 30 or later, or Java 1.7

  • Windows, Suse Linux, or Redhat Linux

Learn About JDBC

Before developing applications that use JDBC, learn more about the Microsoft JDBC 4.0 Driver for SQL Server.

  • For more information about the driver, see Microsoft JDBC Driver for SQL Server on MSDN.

  • For the Books Online driver documentation, see Microsoft SQL Server JDBC Driver 4.0 on MSDN.

Connection String Keywords

The following table lists the supported connection string keywords for connecting to SQL Server PDW by using Microsoft JDBC 4.0 Driver for SQL Server. This is a subset of keywords supported for connections to SQL Server.

For a full description of each string keyword, connection string creation information, and usage of the APIs, see Setting the Connection Properties (SQL Server 2012) on MSDN.

Property | Type | Default | PDW Support
applicationName | String [<=128 char] | null | Yes
databaseName, database | String [<=128 char] | null | Yes
Encrypt | boolean [“true”|”false”] | false | Yes
failoverPartner | String | null | No
hostNameInCertificate | String | null | No
instanceName | String [<=128 char] | null | No
lastUpdateCount | boolean [“true”|”false”] | true | No
lockTimeout | int | -1 | No
loginTimeout | Int [0…65535] | 15 | Yes (client side)
packetSize | int [-1|0|512…32767] | 8000 | Yes
Password | String [<=128 char] | null | Yes
portNumber, port | Int [0…65535] | 1433 | Yes
responseBuffering | String [“full”|”adaptive”] | adaptive | Yes (client)
sendStringParametersAsUnicode | boolean [“true”|”false”] | true | Yes
sendTimeAsDatetime | boolean [“true”|”false”] | true | Yes
serverName, server | String | null | Yes
username, user | String [<=128 char] | null | Yes
trustServerCertificate | boolean [“true”|”false”] | false | Yes This property only affects the server SSL certificate validation if and only if the encrypt property is set to “true”.
trustStore | String | null | Yes
trustStorePassword | String | null | Yes
workstationID | String [<=128 char] | <empty string> | Yes
xopenStates | Boolean [“true”|”false”] | false | No

[TOP]

Supported Data Types

All of the data types that SQL Server PDW supports are also supported when connecting to SQL Server PDW by using the Microsoft JDBC Driver 4.0 for SQL Server. For a list of the supported data types for SQL Server PDW, see Data Types (SQL Server PDW)

[TOP]


Informatica Connector User Guide (SQL Server PDW)

[image]

Informatica Connector User Guide (SQL Server PDW)
Send Feedback

This guide provides a general overview of Microsoft SQL Server PDW Informatica Connector as well as specific information for installing and integrating this product into your Informatica environment.

Summary

The SQL Server PDW Informatica Connector integrates PDW with Informatica. It is an Informatica Power Exchange based connector that allows a user to use a SQL Server PDW table as a source or a target within Informatica mappings. This connector also allows a user to use the SQL Server PDW bulk loader functionality from Informatica and load a flat file data into PDW.

This guide assumes that the people responsible for integrating SQL Server PDW for Informatica are experienced users of the various Informatica applications including the Informatica Administrator, Informatica PowerCenter Repository Manager, Informatica PowerCenter Designer, and Informatica PowerCenter Workflow Manager. It does not explain basic operation of the Informatica applications. For questions about the Informatica applications, consult the Informatica documentation.

Only 64-bit connectors are supported.

Supported Versions

Informatica Connector | Informatica PowerCenter | Driver
SQL Server 2008 PDW | 8.6.1 | SQL Server PDW ODBC client SQL Server Native Client 10.x
SQL Server 2008 PDW | 9.0.1 | SQL Server Native Client 10.x
SQL Server 2012 PDW | 9.0.1, 9.5.1 | SQL Server Native Client 11.x

Plug-in Components

Client Component

Provides a GUI to the user in order to create SQL Server PDW as a source or target and import this table definition within Informatica designer.

Server Component

If any mapping is using PDW as source or target then the server component will communicate with SQL Server PDW. It can read data from a PDW table and sends it to Informatica or it can take data from Informatica and write it to a PDW target. It also provides bulk loading functionality.

Repository Component

This plug-in configuration xml file contains all SQL Server PDW supported data types associations with ODBC data types and other metadata related info. It also has Vendor ID and Plug-in ID for this connector. The SQL Server PDW Informatica Connector supports all data types supported by PDW.

In This Section

This user guide has the following sections:

  • Install the Informatica Connector (SQL Server PDW)

  • Using SQL Server PDW tables as Informatica Sources and Targets (SQL Server PDW)

  • Using Informatica to create SQL Server PDW connections in workflow manager (SQL Server PDW)

  • Creating Sessions and Workflows in Informatica (SQL Server PDW)

  • Troubleshooting the SQL Server PDW Informatica Connector (SQL Server PDW)

  • Uninstalling the Informatica Connector Server (SQL Server PDW)


Install the Informatica Connector (SQL Server PDW)

[image]

Install the Informatica Connector (SQL Server PDW)
See Also Send Feedback

The Informatica Connector server should be configured on the Informatica Server 64 bit computer. The Informatica Connector client should be configured on the 64-bit Informatica Client computer.

Installation Steps

Client and Server Installation Steps

  1. Run the installer msi. Installer will check the registry. On failure the installer will abort. On success it will show the Welcome dialog.

  2. Click Next to proceed to the license agreement.

  3. Accept to enable the Next button. Click Next to proceed.

  4. Complete the Information dialog box, ask for confirmation, and click on Install to proceed with installation process.

  5. On successful installation, the installer will show the installation is complete screen. Click on Finish to complete the process.

See the related topic in this guide for additional post installation configuration tasks.

  • Using SQL Server PDW tables as Informatica Sources and Targets (SQL Server PDW)

  • Using Informatica to create SQL Server PDW connections in workflow manager (SQL Server PDW)

  • Creating Sessions and Workflows in Informatica (SQL Server PDW)

Registering and Verifying a Plug-in

Register a Plug-in

  1. Open the Informatica Administrator at https://infa_server_host_name:port_number. The default https port is 8443.

  2. Connect to the desired repository.

  3. In the Domain Navigator pane, right-click the repository.

  4. On the Properties tab, click Edit.

  5. In the Edit Repository Properties section, in the Operating Mode box, select Exclusive.

  6. On the Plug-Ins tab, browse and select the SQLServerPDW _Plugin.xml. (The path of this xml file is normally where the Informatica client is installed. The default path is C:\Informatica\<version>\clients\PowerCenterClient\client\Microsoft.)

  7. Click Register, and complete the dialog box.

  8. After registration, change the operating mode to normal.

Verifying the Create SQL Server PDW Source and Target in Menu Item

  1. In the PowerCenter Designer, open the appropriate repository.

  2. On the Tools menu, click Source Analyzer. This opens the Source Analyzer window, and changes some of the menu options.

  3. On Sources menu, confirm that the menu item Import from SQL Server PDW is now available.

See Also

Tasks

Uninstalling the Informatica Connector Server (SQL Server PDW)


Using SQL Server PDW tables as Informatica Sources and Targets (SQL Server PDW)

[image]

Using SQL Server PDW tables as Informatica Sources and Targets (SQL Server PDW)
See Also Send Feedback

By using the SQL Server PDW Informatica Connector, Informatica can use PDW tables as source and target tables. The following procedures configure PDW tables to appear in the Informatica Designer.

Make sure that Informatica and Microsoft SQL Server PDW Informatica Connector are successfully installed on the specified system and the plug-in is registered in the Informatica Admin Console.

Import Table from SQL Server PDW into Informatica Source

The following steps configure PDW as a source.

Informatica select DSN Source

  1. Open the Informatica PowerCenter Designer and connect to the desired repository.

  2. On the Tools menu, click Source Analyzer.

  3. On the Sources menu, click Import from Microsoft SQL Server PDW.

  4. Select a SQL Server data source.

  5. Enter a valid Username.

  6. Enter a valid Password.

  7. Click Connect.

  8. Click the + sign to expand Tables.

  9. Select the tables which you want to import.

  10. Click OK.

  11. Under the Repository folder, Sources section, verify SQL Server PDW.

Import Table from PDW into Informatica Target

The following steps configure PDW as a target.

Informatica select DSN Target

  1. Open the Informatica PowerCenter Designer and connect to the desired repository.

  2. On the Tools menu, click Target Designer.

  3. On the Targets menu, click Import from Microsoft SQL Server PDW.

  4. Select a SQL Server data source.

  5. Enter a valid Username.

  6. Enter a valid Password.

  7. Click Connect.

  8. Click the + sign to expand Tables.

  9. Select the tables which you want to import to the target.

  10. Click OK.

  11. Under the Repository folder, Targets section, verify the Imported tables.

Adding an Update Strategy Transformation

The user must specify a column used as a primary key if the update strategy transformation is going to be used in the mapping.

Informatica select a primary key column

  1. Right-click a table and select Edit.

  2. In the Column Name section, select the column that is a primary key. In the Key Type section, select PRIMARYKEY for that column.

See Also

Tasks

Install the Informatica Connector (SQL Server PDW)
Using Informatica to create SQL Server PDW connections in workflow manager (SQL Server PDW)
Creating Sessions and Workflows in Informatica (SQL Server PDW)


Using Informatica to create SQL Server PDW connections in workflow manager (SQL Server PDW)

[image]

Using Informatica to create SQL Server PDW connections in workflow manager (SQL Server PDW)
See Also Send Feedback

Informatica can make connections to SQL Server PDW for use in relational queries, or for loading data.

How to create a relational connection

Informatica Relational Connection dialog box

  1. Open the Informatica Workflow Manager and connect to the desired repository.

  2. On the Connections menu, click Relational.

  3. In the Select Type box, select Microsoft SQL Server PDW, and then click New.

  4. In the Name box, provide a name for this connection.

  5. In the User Name and Password boxes, provide the credentials for connecting to PDW.

  6. Optionally, select a Code Page.

  7. In the Attributes area, specify the connection string values.

    • Host Name: The name or IP address of the SQL Server PDW.

    • Database Name: Name of the database.

    • Port Number: 17001.

  8. Click OK.

How to create a loader connection

Informatica Loader Connection dialog box

  1. Open the Informatica Workflow Manager and connect to the desired repository.

  2. On the Connections menu, click Loader.

  3. In the Type box, select Microsoft SQL Server PDW Loader, and then click New.

  4. In the Name box, provide a name for this connection.

  5. In the User Name and Password boxes, provide the credentials for connecting to PDW.

  6. Optionally, specify the Code Page.

  7. In the Attributes area, specify the connection string values.

    • Host Name: The name or IP address of the SQL Server PDW.

    • Database Name: Name of the database.

  8. Optionally, specify additional attributes. The following attributes and values are typical.

    • Batch Size: Batch size indicates the number of rows to be written to each intermediate output file. Once created, these output files are loaded to the PDW in parallel (as they are created) by the Bulk Loader component. The default value is 5000. The user should configure this according to number of input rows to be loaded in the PDW. Very large values are not recommended because of the resources required for I/O operations.

    • Use SSL: If this option is set to true (the default value), data will encrypted before it is sent to PDW. This is more secure but slows down the transfer.

  9. Click OK.

See Also

Tasks

Install the Informatica Connector (SQL Server PDW)
Using SQL Server PDW tables as Informatica Sources and Targets (SQL Server PDW)
Creating Sessions and Workflows in Informatica (SQL Server PDW)


Creating Sessions and Workflows in Informatica (SQL Server PDW)

[image]

Creating Sessions and Workflows in Informatica (SQL Server PDW)
See Also Send Feedback

Use the Informatica Workflow Manager to create workflows and sessions, and to configure the session extensions.

Creating a Workflow

  1. Open the Informatica Workflow Manager and connect to the desired repository.

  2. Click on Workflow Designer, click Workflow, click Designer, and then click Create.

Creating a session

  1. Open the Informatica Workflow Manager and connect to the desired repository.

  2. Click on Task Developer, click Tasks, and then click Create.

  3. Assign a mapping.

Configuring sessions

After creating a session, use the Mapping tab to configure properties of the session.

  1. Click on Task Developer, click Tasks, select a task, and then click Edit.

  2. In the Edit Tasks page, click the Mapping tab.

Configure the source connection

  1. In the left pane, expand Sources, and select a source.

  2. In the Connections area, configure the connection.

    Type

    Select Relational.

    Value

    Provide an appropriate connection name.

    Connections

    Select Microsoft SQL Server PDW.

APS Informatica Session Extensions dialog box

Configure the target connection

  1. In the left pane, expand Targets, and select a target.

  2. In the Writers area, select the desired Writer (PDW Connector Writer or Loader).

  3. In the Connections area, configure the connection.

    Type

    Select Relational or Loader.

    Value

    Provide an appropriate connection name.

    Connections

    Select Microsoft SQL Server PDW.

APS Informatica Session Target dialog box

Setting Relational Connection

Additional information about relational connection session level properties.

In the Connections area, in the Type box, select Relational.

Loader connection sessions

Additional information about loader connection session level properties.

In the Properties area, for the Output file directory and Output filename attributes, enter a directory and filename.

This file will be automatically deleted after the execution of the workflow on both success and failure.

Output file directory

The output file directory specifies the location of the intermediate files used during the bulk loading of the data. The source data is written to a flat file in string format and then loader component loads data from these files to the SQL Server PDW. The data from multiple files is loaded in parallel to boost performance. The default directory is Informatica's target file directory.

Output filename

Name of the output file. The default value is the name of the target table. During runtime, each new file name is appended with a number.

Load mode

Indicates how data is handled during the load process. Options for mode are append, fastappend, upsert, or reload. The default mode is append.

  • In append mode, data is inserted at the end of existing table data.

  • In fastappend mode, the data is inserted at the end of the existing table. In addition, fastappend loads directly into the final destination table and does not use a staging table. fastappend requires multi-transaction mode (-m).

  • In upsert mode, if the row does not already exist in the table (based on a given value) the row will be inserted. If the row does already exist in the table, the existing entry will be updated.

  • In reload mode, the loader component truncates the destination table, then inserts into the destination table with data from the staging table.

Use multiple transactions

Specifies that multiple transactions can be used inside the engine while loading the data. By default, this is not enabled and the load will be performed as a single transaction (that may require more time, but can be rolled back). Setting this option enables loads to go faster, but is not transactionally safe. This option should only be used for initial loads of tables where performance is a major concern. In the event of a load failure during initial table loads, the destination tables can be manually dropped by the user. Multi-transaction load is only permitted if the destination table is a distributed table; if -m is used with the destination table being a replicated table, the load is aborted and an error message is returned to the user.

See Also

Tasks

Install the Informatica Connector (SQL Server PDW)
Using SQL Server PDW tables as Informatica Sources and Targets (SQL Server PDW)
Using Informatica to create SQL Server PDW connections in workflow manager (SQL Server PDW)


Troubleshooting the SQL Server PDW Informatica Connector (SQL Server PDW)

[image]

Troubleshooting the SQL Server PDW Informatica Connector (SQL Server PDW)
See Also Send Feedback

This section helps you identify problems related to the SQL Server PDW Informatica connector. Problems may also arise from your Informatica applications, so it may be helpful to consult the Informatica Power Center documentation as well.

Before troubleshooting, check that Informatica and the SQL Server PDW Informatica Connector are successfully installed on the specified system and the plug-in is registered in the Informatica Admin Console.

The SQ Property Page

Use the following steps to set the value of the SQ property page.

Informatica SQ Properties

  1. Open the Informatica Mapping Designer and connect to the desired repository.

  2. Navigate to required mapping with the Microsoft SQL Server PDW sources.

  3. Right-click the mapping, and then click Edit Application source qualifier.

  4. On the SQ Properties tab, enter the desired properties.

    User Defined Join

    Specify a join condition for multiple tables.

    No. of Sorted Ports

    Specify the number of sorted ports which must be less than the total number of columns.

    Select Distinct

    Select distinct values from the table.

    Source Filter

    Restrict the rows returned by providing a filter condition.

  5. Click OK.

Override Query at Session Level

Users can create a customized query at mapping level. Use this feature to override those queries.

Informatica Query Override page

  1. Open the Informatica Workflow Manager and connect to the desired repository.

  2. Navigate to the required session.

  3. Right-click the session, and then click Edit session.

  4. In the Edit Tasks dialog box, on the Mapping tab, in the left pane, expand Sources, and then select a table.

  5. In the Readers area, click SQL Server PDW Reader.

  6. In the Connections area, in the Type box, select Relational.

  7. In the Value box, select your SQL Server PDW Relational Connection.

  8. On the Properties section, enter the desired attributes and values. Query attributes entered here will override the query of SQ Properties tab.

    User Defined Join

    Specify a join condition for multiple tables.

    Number of Sorted Ports

    Specify the number of sorted ports which must be less than the total number of columns.

    Select Distinct

    Select distinct values from the table.

    Source Filter

    Restrict the rows returned by providing a filter condition.

  9. Click OK.

Importing Sample Workflows

Use the following steps to import sample workflows into Informatica PowerCenter. These steps are the same for different versions of Informatica.

Before importing, create sample tables as directed in the text file Table Scripts for PDW.txt. Then insert data into these tables using insert scripts provided in the files Insert Script - connector_sample bulk_load_ source.txt and Insert Script - connector_sample.txt files. These scripts are included in the SQL Server PDW Informatica Connector msi and installed with the connector. The default location for the files is C:\Informatica\9.5.1\clients\PowerCenterClient\client\Microsoft\Sample.

  1. Open the Informatica Repository Manager and connect to the desired repository.

    If you are already in the Informatica PowerCenter Designer, you can access the Repository Manager by clicking on the R icon on the toolbar.

    Informatica Sample A

  2. Click on the folder in which the workflows are to be imported.

  3. On the Repository, click Import Objects.

    Informatica Sample B1

  4. Select the XML file installed at INFA_HOME\client\Microsoft\Sample, and then click Next. You must choose the appropriate file for your version of Informatica. (INFA861_Sample_Workflows.xml for INFA861 and INFA901_Sample_Workflows.xml for INFA901.)

    Click Add All, and then click Next.

    Informatica Sample B2

  5. In the Destination folder box, select the folder where you want to import workflows, and then click Next.

    Informatica Sample B3

  6. In the Sessions Configurations area, select default_session_config, and then select the Reuse checkbox.

    Informatica Sample B4

  7. Click Close to return to the import wizard.

  8. Review the status, and then click Import to import the workflows in the selected folder.

    Informatica Sample B5

  9. Informatica will import all the sample workflows into the repository. The user should see a final status message as Completed importing selected objects.

    Informatica Sample B6

  10. After importing, copy the flat file source names pdw_data.txt from INFA_HOME\client\Microsoft\Sample to the INFA_HOME\server\infa_shared\SrcFiles folder.

  11. When this is completed, you can run the imported mappings. You must make sure that connections are properly created and set to the mappings before running them.

See Also

Tasks

Install the Informatica Connector (SQL Server PDW)
Using SQL Server PDW tables as Informatica Sources and Targets (SQL Server PDW)
Using Informatica to create SQL Server PDW connections in workflow manager (SQL Server PDW)
Creating Sessions and Workflows in Informatica (SQL Server PDW)


Uninstalling the Informatica Connector Server (SQL Server PDW)

[image]

Uninstalling the Informatica Connector Server (SQL Server PDW)
See Also Send Feedback

Use the following steps to uninstall the SQL Server PDW Informatica Connector. These instructions apply to all versions of PDW and all versions of Informatica.

Uninstalling the Informatica Connector Server

  1. Open the Informatica Administrator and connect to the desired repository.

  2. In the Domain Navigator pane, right-click the repository.

  3. On the Properties tab, click Edit.

  4. In the Edit Repository Properties section, in the Operating Mode box, select Exclusive.

    Uninstall Informatica step 1

  5. Restart the service when prompted.

  6. In the section for your repository, select Plugins, select the PDW plugin, and then click Unregister Plug-in.

    Uninstall Informatica step 2

  7. In the Windows Control Panel, Programs and Features, Uninstall or change a program page, select the Microsoft SQL Server PDW Connector for Informatica Server for your version, and then click Uninstall.

See Also

Tasks

Install the Informatica Connector (SQL Server PDW)

Concepts

Informatica Connector User Guide (SQL Server PDW)


Design (SQL Server PDW)

[image]

Design (SQL Server PDW)
Send Feedback

Principles for designing SQL Server PDW databases.

  • Minimize Data Skew When Choosing a Distribution Column (SQL Server PDW)

  • Minimize Data Movement When Choosing a Distribution Column (SQL Server PDW)

  • None of the Existing Columns Are a Good Choice for the Distribution Column (SQL Server PDW)


Minimize Data Skew When Choosing a Distribution Column (SQL Server PDW)

[image]

Minimize Data Skew When Choosing a Distribution Column (SQL Server PDW)
Send Feedback

This article describes best practices for choosing a distribution column that minimizes data skew for a SQL Server PDW distributed table. Choosing a good distribution column is an important aspect of achieving storage efficiency and highly performant queries. This is possibly the most important design choice you will make in SQL Server PDW.

The best choice for a distribution column depends on the data characteristics and the queries you intend to process. The goal is to choose a distribution column that minimizes data skew among the compute nodes and minimizes data movement during query processing. These are both important and sometimes competing considerations.

Best Practices

The following best practices facilitate storing a similar number of rows in each distribution, which minimizes data skew.

Choose a column with high cardinality that does not have data that is heavily “skewed” – where a disproportionately large number of rows are associated with one or a very few column values.

No distribution key value should have a disproportionate number of duplicates in comparison to the other distribution key values. For example, a common scenario in data warehousing is storing the clickstream data for a web site. In clickstream data, a disproportionate number of clicks can occur on only a few of the web site pages such as the Home page. If you use the web page URL as the distribution key, the table rows for each page will all be stored in the same distribution. If 30% of the clicks occur on the Home page, then 30% of the rows will all be stored in one distribution.

Conversely, a distribution column can contain values with a large number of duplicates provided there are a lot of values with a large number of duplicates. Hence, high cardinality is important. For example, if you have 10,000 values with high counts and 100,000 values with low counts, all of the values can be distributed evenly across all of the distributions.

Here's another example for why high cardinality is important. Suppose you have 80 distributions and are storing sales receipts for only 100 stores. If you use the store ID number for the distribution key, it is likely that there are not enough stores to spread the table rows evenly among the distributions. Even in the best case, where each distribution stores sales receipts for either 1 or 2 stores, some distributions would have twice as many table rows than other distributions. More likely, with only 100 different key values for 80 distributions, many distributions will end up being completely empty.

Choose a column that does not allow NULL, or where NULL values are as rare as any other column value.

If a distribution key can be NULL, all rows with the NULL value will be stored in the same distribution. This is potentially a huge portion of rows that are skewed to the same distribution. For example, if 40% of the tables rows have NULL for the distribution key, then 40% of the table rows will be stored in the same distribution.

Choose an Integer column when possible

Integer columns are preferable for the distribution column rather than varchar, decimal and datetime columns.

Avoid using datetime columns unless time is accurate to at least the minute.

A datetime column might or might not be a good choice for the distribution column. For example, if a datetime column is accurate to the day, and typical queries access a single day or small number of days, most of the work for such queries will be restricted to a single distribution or small number of distributions, defeating the benefits of the MPP architecture. However, if the datetime column is accurate to the minute or fraction of a second, data for even a single day query will be evenly spread across all distributions, and the column can therefore be a good choice for the distribution column.

Avoid choosing a key that is used as a single value in a WHERE clause.

Some of the most popular queries in a sales data warehouse are queries on yesterday’s data. For example, if your query has WHERE DATE = ‘20100809’ and the table is distributed on the DATE key, the entire query will be run within one distribution. This is because all rows in the table that have the same date will be stored in the same distribution. As a result, you won’t get the benefits of parallelism.

Background

Data skew is a condition that occurs when the rows of a distributed table are not spread uniformly across all of the distributions. When the number of rows in each distribution varies significantly, this can negatively impact query performance.

Avoiding data skew means you want to avoid having a disproportionately large number of table rows in one distribution in comparison to the number of rows in other distributions. Since queries rely on work being performed on all distributions, even if queries finish fast in smaller distributions, you will still need to wait for the queries to finish on larger distributions before you can get the query result.

How to determine if data skew is an issue

Don’t worry about making a mistake when you choose the distribution column. It’s easy to re-create the table using another distribution column by using the CREATE TABLE AS SELECT (CTAS) statement.

To test your workloads with different distribution keys, you can create copies of the same table and assign a different distribution column to each table. You can then test your workloads against each table to see which distribution key gives the best overall performance. This type of iterative design and experimentation works well with SQL Server PDW architecture.

To see the counts of rows per distribution in a table, use the command:

DBCC PDW_SHOWSPACEUSED (dbo.<tablename>)

For easy analysis, copy the results into a spreadsheet. Examine the results to determine whether any distributions contain disproportionately larger or smaller numbers of rows than the average. Rowcounts skewed 30% or higher could have a negative impact on query performance. The amount of acceptable data skew depends on your workload and performance requirements.

You should absolutely choose a different distribution strategy if several distributions show 0 rows (for a large table), or if a small number of distributions show an order of magnitude more rows than the others.

Before you make changes to minimize data skew

Before you decide to take further action, make sure you really do have a skew problem and understand what it will do to the overall workload on the appliance. Generally, a 10% difference in the number of rows among distributions is acceptable. The amount of acceptable data skew depends on your workload and performance requirements.

We have seen that 10-20% skew can be tolerable in PDW when considering the big scheme of things where many queries are going through the system concurrently. Obviously, single-run queries that are affected by the skew will naturally run slower on the skewed distributions, but before you declare defeat, make sure that your customer (end user, not the DBA!) isn’t unhappy with performance.

Just because data may be skewed on one or a few distributions doesn’t spell disaster. Look at the customer’s queries and ensure that they actually will be affected.

Naturally, if the queries are doing full distribution scans, then skew will affect them… but this is very rare in the real world. If the distributions are also partitioned (e.g. by date) and the queries are restricted by date, then a good (possibly mixed-grain) partitioning strategy can work wonders. Yes, the queries may still run a bit slower on the skewed partitions, but the performance is relative to the scan rate so it may not be that much slower. Testing will tell for sure.


Minimize Data Movement When Choosing a Distribution Column (SQL Server PDW)

[image]

Minimize Data Movement When Choosing a Distribution Column (SQL Server PDW)
Send Feedback

This article describes best practices for choosing a distribution column that minimizes data movement for a SQL Server PDW distributed table. Choosing a good distribution column is an important aspect of achieving storage efficiency and highly performant queries. This is possibly the most important design choice you will make in SQL Server PDW.

The best choice for a distribution column depends on the data characteristics and the queries you intend to process. The goal is to choose a distribution column that minimizes data skew among the compute nodes and minimizes data movement during query processing. These are both important and sometimes competing considerations.

Best Practices

The following best practices describe how to choose a distribution column that minimizes data movement for your workloads.

1. Choose distribution keys that will be join keys

Choose a distribution column that is one of the join keys when queries will be joining multiple large distributed tables. This improves query performance by keeping the processing local to each distribution instead of shuffling rows amongst the distributions before processing the query. If possible, use sample queries to determine which join keys they contain.

If you have a workload that is joining multiple large historical tables together (e.g. fact table to fact table joins), you will avoid data movement if you can join the distributed tables on their distribution keys.

2. If possible, change an incompatible join to a compatible join by adding a predicate that uses the distribution key

In some cases, the results of a query are not altered by adding another predicate that uses the distribution key. Although this predicate is not necessary logically, it is necessary for performance. If the distribution key is in the join condition, data shuffling is not needed and each distribution can perform its portion of the join locally.

For example, imagine two tables, A and B, distributed on Postal Code, but joined together on a unique key, Address_ID. Adding the additional predicate A.Postal_Code = B.Postal_Code will not change the meaning of the query because postal code depends on the Address_ID. In other words, the following two predicates return the same results:

  • WHERE ( A.Address_ID = B.Address_ID )

  • WHERE ( A.Address_ID = B.Address_ID_ID AND A.Postal_Code = B.Postal_Code )

However, the second predicate will perform more efficiently because it does not require data shuffling.

3. Design your schema to encourage compatible aggregations

The best performance for aggregations occurs when the distribution column is used in the GROUP BY clause (compatible aggregation). The GROUP BY operation is performed locally on all the distributions, without requiring a shuffle move among the distributions or a partition move so the Control node can perform aggregations.

In contrast, the worst aggregation performance occurs when the distribution column is not used in the GROUP BY clause (incompatible aggregation). Each distribution moves all the rows that are part of the GROUP BY operation to the Control node. This leads to a bottleneck because the Control node must perform all the GROUP BY operations.

4. Optimize for a key frequently used in Distinct Count

For example, if SELECT COUNT (DISTINCT Customer_ID) FROM T WHERE ... is used frequently, there will be performance benefits if Customer_ID is the distribution key.

5. Choose a column with static values

Remember, that you cannot update values in the distribution column. If you need to update a value, you can delete the row and insert a new row with the updated value.


None of the Existing Columns Are a Good Choice for the Distribution Column (SQL Server PDW)

[image]

None of the Existing Columns Are a Good Choice for the Distribution Column (SQL Server PDW)
See Also Send Feedback

This article describes best practices for choosing a distribution column when none of the distributed table columns seem to be a good choice. This applies to distribution tables for SQL Server PDW.

Best Practices

Sometimes none of the columns in the table seem to be a good choice for the distribution columns. That’s okay. These are recommendations for choosing the distribution column when that happens.

1. Look for less obvious columns, like those with a timestamp.

If you cannot find an “obvious” uniform column to distribute on, then look for less obvious columns like those with a timestamp. While it is true that we want to be careful about distributing on a chronological column (such as DATE), timestamps that go down to the millisecond have proven to be excellent candidates for distribution because in the business/scientific world data arrive in a continuous flow. They already exist in the table and can easily be tested with no change to ETL/ELT. However, remember that this strategy will probably result in distribution-incompatible joins.

2. Consider creating a new column that combines two columns

If there are no obvious columns and you don’t have a good timestamp column, then consider introducing a new column that combines two columns together to provide a uniform distribution (a composite distribution). This will require loading a sample amount of data into the “original table” and then performing data analysis, but it won’t take you very long to determine if you can merge two columns into a distribution column. Be careful not to make your composite column too large since distributed tables are the largest in the system, you don’t want to take up too much space. Also be sure to choose columns that cannot be updated since you won’t be able to easily redistribute the rows that change (you cannot update a distribution column). Obviously, this strategy will affect your ETL/ELT processing, but the changes need not be very intrusive and are easily implemented.

3. Consider using a unique key or increasing integer

If no other effective distribution keys are present in a table that provide an even distribution of data, consider using a unique key (if one exists) or even an increasing integer value associated with each row, generated during data loading operations.

You can introduce an integer column in the ETL/ELT process or by using a ROW_NUMBER function. This guarantees a uniform distribution, but remember all queries will result in distribution-incompatible joins. You can test your hypothesis by using CTAS and the ROW_NUMBER function to create a sample table, run a representative sampling of the customer’s queries and see what the DSQL plans look like (i.e. look at the number of shufflemove steps and drill into them to see the number of rows re-distributed per distribution).

4. As a final resort, consider redesigning the schema

As a final resort, consider redesigning the schema entirely to make it more MPP friendly. I can honestly say that, in all my travels in the last 7 years, I’ve only had to do this once, but the client had an impossibly bad design to begin with.

See Also

Concepts

Minimize Data Skew When Choosing a Distribution Column (SQL Server PDW)
Minimize Data Movement When Choosing a Distribution Column (SQL Server PDW)


Query (SQL Server PDW)

[image]

Query (SQL Server PDW)
See Also Send Feedback

To write and tune queries, SQL Server PDW provides several tools and processes, which are outlined in the following sections.

Writing Queries

To query your SQL Server PDW databases, you can use the SQL Server PDW SQL language. Queries can be submitted to SQL Server PDW either through SQL Server Data Tools for Visual Studio, or through the sqlcmd command-line tool. For more information about how to install and use these tools, see the following:

  • Client Tools and Applications (SQL Server PDW)

To use the SQL language statements, the SQL Server PDW system views, and the SQL Server views, see the following reference sections:

  • SQL Reference (SQL Server PDW)

  • System Views (SQL Server PDW)

For information on using prepared statements, see the following:

  • Running Prepared Statements (SQL Server PDW)

Tuning Queries

To view the query plan, use the EXPLAIN command. This will allow you to see what steps the query will process without running the query. For more information, see the following:

  • Understanding Query Plans (SQL Server PDW)

  • EXPLAIN (SQL Server PDW)

Monitoring Queries

You can monitor active and recent queries to see the processing time required for each step and other metadata. For more information, see the following:

  • Monitoring Active Queries (SQL Server PDW)

See Also

Concepts


Understanding Query Plans (SQL Server PDW)

[image]

Understanding Query Plans (SQL Server PDW)
Send Feedback

A SQL Server PDW query plan is the translation of sequential SQL statements into an execution plan that will run in parallel on the SQL Server PDW appliance. Use the query plan to understand how your query will be processed and find ways to optimize performance.

Use the EXPLAIN command to generate a query plan without submitting the query for execution.

Contents

  • Understanding the Query Process

  • DMS Query Plan Operations

  • SQL Query Plan Operations

  • Other Query Plan Operations

Understanding the Query Process

User-submitted SQL queries are processed by the Control node. The Control node engine parses the query and creates a query plan that defines the sequence of operations it will use to run the query on the appliance. The Control node distributed query plan operations run serially. When a query plan operation uses multiple parallel operations, the SQL Server PDW engine waits for all parallel operations to complete before starting the next distributed query plan operation.

Parallel Query Processing

And this is the query execution sequence:

SQL Server PDW Query Execution Sequence

[TOP]

DMS Query Plan Operations

A Data Movement Service (DMS) operation is a SQL Server PDW query plan operation that moves data among the nodes in the appliance. The data is always moved into temporary tables. The permanent data in user databases remains intact during DMS operations and is not removed from the original location in the appliance.

Moving data ensures that data exists on the correct nodes at the correct time to run queries in the MPP Shared Nothing architecture. For example, DMS operations are required if the data for a SQL join is not collocated on each Compute node, or data needs to be moved from the Compute nodes to the Control node for an aggregation operation.

SQL Server PDW query plans contain the following DMS operations.

Operation Name

OpType Name for EXPLAIN

Description

BroadcastMoveOperation

BROADCAST_MOVE

Moves distributed data into a replicated table. This operation is frequently used when running a distribution incompatible join.

To perform this operation, each distribution broadcasts its rows to all other Compute nodes. The result is that each Compute node uses its own rows plus all the rows it receives from the other Compute nodes to create a replicated table.

CopyOperation

CopyOperation

Copies data from SQL Server PDW to a SMP SQL Server database. This operation is used only when running the CREATE REMOTE TABLE AS SELECT (SQL Server PDW) statement. It is always preceded by a RemoteOperation.

DistributedReplicatedTableMoveOperation

DISTRIBUTE_REPLICATED_TABLE_MOVE

Copies a replicated table from one Compute node to all other Compute nodes. This operation guarantees data consistency among all copies of a replicated table after a non-deterministic query operation. For example, it might be necessary to perform this operation after an Insert, Update, Delete, or CTAS statement on a replicated table.

MoveOperation

MASTER_TABLE_MOVE

Moves data from the Control node to all Compute nodes. This operation is used to copy data into a replicated table.

PartitionMoveOperation

PARTITION_MOVE

Moves data from a distributed table to a single table on the Control node. This operation is used for aggregation operations on the Control node.

ReturnOperation

RETURN

Sends query results from the Control node to the user who submitted the query. This is the final operation for a parallel query plan.

ShuffleMoveOperation

SHUFFLE_MOVE

Redistributes a distributed table. The redistributed table has a different distribution column than the original distributed table. This might be used to when running incompatible joins or incompatible aggregations.

To perform this operation, DMS moves each row to the correct Compute node according to distribution column of the destination table. Rows that are already stored on the correct Compute node are not copied during this operation.

TrimMoveOperation

TRIM_MOVE

Moves a replicated table to a distributed table.

To perform this operation, data does not need to be copied from one Computer node to another because all rows are already stored in the replicated table on each Compute node. Instead, each Compute node distributes only the rows that belong in its distributions.

DSQLCallBackOperation

(Not visible via EXPLAIN)

Redistributes distributed tables as part of a database restore. This is used only when a restore operation is restoring a database to a destination appliance that has more Compute nodes than the source appliance.

[TOP]

SQL Query Plan Operations

A SQL operation performs an action on a database or object. Each SQL operation runs on a specific set of nodes. For example, it can run on the Control node, some of the Compute nodes, all of the Compute nodes, or a remote SQL Server instance.

Operation Name

OpType Name for EXPLAIN

Description

OnOperation

ON

Performs an action on a database or object. The OnOperation can be performed after DMS operations have completed, or it can be used to run a query that does not require data movement. Each OnOperation specifies which node or nodes that will perform the action.

For example, the OnOperation performs DDL statements such as Create Table, Create Index, Drop Table, Drop View, etc.

RemoteOperation

RemoteOnOperation

Performs an action on a SMP SQL Server database or table. This is similar to OnOperation, but is only used when running the CREATE REMOTE TABLE statement.

[TOP]

Other Query Plan Operations

Operation Name

OpType Name for EXPLAIN

Description

MetaDataCreateOperation

MetaDataCreate_Operation

Stores data into or retrieves data from a system view. For example, this operation is used to perform a SELECT from a system view.

RandomIDOperation

RND_ID

Creates a unique name. This operation is used to generate random names for temporary objects that get created as part of the query plan. The RandomIDOperation is always run on the Control node.

Temporary tables are named q_<RandomID>. The query plan might use q_temp_partition1 to refer to a temporary name before it is generated.

[TOP]

External Operations for Reads

Operation Name | OpType Name for EXPLAIN | Description
HadoopBroadcastOperation | ExternalBroadcastMove |
HadoopRoundRobinOperation | ExternalRoundRobinMove |
HadoopShuffleOperation | ExternalShuffleMove |

[TOP]

External Operations for MapReduce

Operation Name | OpType Name for EXPLAIN | Description
HadoopJobOperation | HadoopOperation |
HdfsDeleteOperation | HadoopFileOperation |

[TOP]

External Operations for Writes

Operation Name | OpType Name for EXPLAIN | Description
ExternalExportControlOperation | ExternalExportControlMove |
ExternalExportDistributedOperation | ExternalExportDistributedMove |
ExternalExportReplicatedOperation | ExternalExportReplicatedMove |

[TOP]


Understanding Query Optimization Statistics (SQL Server PDW)

[image]

Understanding Query Optimization Statistics (SQL Server PDW)
See Also Send Feedback

SQL Server PDW uses statistics to improve the quality of the data movement and Transact-SQL query plans, and therefore to improve query performance.

Contents

  • Basics

  • Differences from SQL Server

  • Recommendations

Statistics Basics

statistics objects

Statistics are objects that contain statistical information about the distribution of values in one or more columns of a table. The query optimizer uses these statistics to estimate the cardinality, or number of rows, in the query result. These cardinality estimates enable the query optimizer to create a high-quality query plan.

In SQL Server PDW, statistics can help improve both the plan for moving data and the plans for running SQL Server PDW queries on the Compute nodes.

For example, on the Compute nodes, the SQL Server query optimizer could use cardinality estimates to choose the index seek operator instead of the more resource-intensive index scan operator, and in doing so improve query performance. On the Control node, the dsql query optimizer could use cardinality estimates to choose a shuffle move instead of a broadcast move, and therefore improve query performance.

Compute node statistics

SQL Server PDW auto-generates statistics on the Compute nodes, and uses them to improve query performance for the local queries that run on the Compute nodes. The Compute node statistics are SMP statistics generated by SQL Server with auto-create and auto-update on. These statistics are not visible to the user through the SQL Server PDW metadata .

For full control over which statistics are generated on the Compute nodes, we recommend creating single column statistics prior to running queries. This enables you to manage the statistics through the statistics DDL statements, and to view the statistics in the system views.

Control node statistics

The Control node merges the Compute node statistics and stores them as a single statistics object. You can call them MPP statistics or merged statistics. SQL Server PDW uses these MPP statistics to estimate the number of rows that need to be moved among the Compute nodes in order to satisfy the Transact-SQL query. Good cardinality estimates allow the dsql query optimizer to choose a high quality plan for data movement operations that reduces the data movement and therefore improves query performance.

The Control node does not auto-create and auto-update single column MPP statistics. You need to do this manually by calling CREATE STATISTICS. When you create statistics on a distributed table, SQL Server PDW creates an SMP statistics object on each Compute node, and one merged statistics object on the Control node. Since all replicated tables have the same statistics, generating merged statistics only applies to distributed tables.

External table statistics

SQL Server PDW supports both full scan and sampled statistics on statistics on external tables.

To create sampled statistics on an external table, SQL Server PDW first determines the sample size by using a method similar to the one used by SQL Server. Then SQL Server PDW imports the sampled rows according to the determined sample size. Only the columns specified in the CREATE STATISTICS statement are imported; the entire row does not get imported. After the data is imported, SQL Server PDW generates SQL Server statistics and stores them on the Control node.

To create full scan statistics, SQL Server PDW creates statistics on all of the rows, using only the columns specified by the CREATE STATISTICS statement. Only the columns specified are imported into SQL Server PDW.

auto-create statistics

For the statistics on the Compute nodes, SQL Server PDW has AUTO_CREATE_STATISTICS set to on. Using this setting, the SQL Server query optimizer, running on each Compute node, generates single-column statistics as necessary to create a high quality SQL Server query plan.

For the statistics on the Control node, the AUTO_CREATE_STATISTICS setting does not apply. Single-column statistics auto-created on Compute nodes do not get merged and stored on the Control node. Since the MPP plan, including data movement operations, is generated on the Control node, the cost-based MPP query optimizer does not auto-create single-column statistics to improve the query plan for data movement operations.

Therefore, to get the benefit of the cost-based MPP query optimizer for data movement and other MPP-related operations, you will need to create statistics on strategic columns of distributed tables by using the CREATE STATISTICS statement.

auto-update statistics

Statistics Differences From SQL Server

Creating Statistics Differences From SQL Server

Creating statistics in SQL Server PDW has a few differences from creating statistics in SQL Server. In SQL Server, you can create statistics in one of the following ways:

  1. Create an index. Statistics are always created for each index.

  2. Create statistics with the CREATE STATISTICS (Transact-SQL) statement. This creates statistics on one or more columns in a table or an indexed view.

  3. Set the AUTO_CREATE_STATISTICS setting to ON. This creates single-column statistics on columns in the query predicate, as needed to create a high quality query plan.

The following table describes the differences between creating statistics in SQL Server and SQL Server PDW.

Method for Creating Statistics | SQL Server | SQL Server PDW
Create a row store index on a table. | Creates SMP statistics on the index. | Creates SMP statistics on each Compute node. Merges the SMP statistics and stores the merged statistics object on the Control node.
Create a column store index on a table | Creates SMP statistics on the first column of the table | Creates SMP statistics on each Compute node. Does not create merged statistics on the Control node
Run the CREATE STATISTICS statement on a table. | Creates statistics on the table. | Creates SMP statistics on the associated tables stored on the Compute nodes. Merges the SMP statistics and stores the merged statistics on the Control node.
AUTO_CREATE_STATISTICS set to ON. | User-configurable. When on, the query optimizer creates SMP statistics on individual columns in the query predicate, as necessary, to improve cardinality estimates for the query plan. These single-column statistics are created on columns that do not already have a histogram in an existing statistics object. | Always on for user databases on the Compute nodes. The query optimizer, running on each Compute node, creates SMP statistics on individual columns in the query predicate, as necessary, to improve cardinality estimates. The single-column statistics are created on columns that do not already have a histogram in an existing statistics object.and is not user-configurable.

TOP

Updating Statistics Differences FROM SQL Server

Using SQL Server, you can update statistics in the following ways:

  1. Update statistics with the UPDATE STATISTICS (Transact-SQL) statement, or a stored procedure that calls UPDATE STATISTICS.

  2. Set the AUTO_UPDATE_STATISTICS setting to ON. With this method, the SQL Server query optimizer determines when statistics might be out-of-date and then updates them when they are needed for a query plan.

The following table describes the similarities and differences between SQL Server and SQL Server PDW for updating query optimization statistics.

Method for Updating Statistics

SQL Server

SQL Server PDW

Set the AUTO_UPDATE_STATISTICS setting to ON.

The SQL Server query optimizer determines when statistics might be out-of-date and then updates them when they are needed for a query plan.

AUTO_UPDATE_STATISTICS is not configurable in SQL Server PDW. All SQL Server databases on the Compute nodes have AUTO_UPDATE_STATISTICS set to ON.

When SQL Server updates statistics on the Compute nodes, as a result of the AUTO_CREATE_STATISTICS setting, SQL Server PDW does not create merged statistics on the Control node.

Run the UPDATE STATISTICS statement.

Updates statistics on a table. The parameters determine whether the update applies to all statistics objects on a table, one statistics object, or one index.

Updates SQL Server statistics on the associated tables stored on the Compute nodes.

Updates the merged statistics on the Control node.

TOP

Recommendations

Creating Statistics

  1. Create statistics on distributed tables, particularly columns in query join predicates. This enables you to gain the benefit of the cost-based query optimizer for data movement and other MPP-related operations.

    • We recommend creating single-column statistics on each column in ORDER BY clauses, join predicates, WHERE clauses, and GROUP BY clauses.

    • For tables with many columns and rows, it usually takes too long to create and update single-column statistics on every column.

  2. Follow the SQL Server guidance for creating and updating statistics. For example, you might have queries for which multi-column statistics will improve performance. For more information, see Statistics on MSDN. (Note, this article refers to a Database Tuning Advisor, which is not available in SQL Server PDW.)

TOP

Updating Statistics

SQL Server automatically updates statistics on the Compute nodes as necessary to improve the SQL Server query plan. However, for distributed tables, SQL Server PDW does not automatically update statistics on the Control node. Using the UPDATE STATISTICS (SQL Server PDW) statement is the only way to update statistics on the Control node.

  1. Update statistics after loads or when significant changes have occurred.

  2. Don’t update statistics “too” often for performance reasons.

    Updating statistics ensures that queries compile with up-to-date statistics. However, updating statistics causes queries to recompile, which can be time-consuming. We recommend not updating statistics too often because there is a performance tradeoff between improving query plans and recompilation time. The specific tradeoffs depend on your application.

  3. Follow the SQL Server guidance for when to update statistics. For more information, see Statistics on MSDN. (Note, this article refers to a Database Tuning Advisor, which is not available in SQL Server PDW.)

TOP

Related Statistics Tasks

Statistics Task | Description
Create statistics. | CREATE STATISTICS (SQL Server PDW)
Update statistics. | UPDATE STATISTICS (SQL Server PDW)
Drop statistics. | DROP STATISTICS (SQL Server PDW)

TOP

See Also


Understanding Compatible and Incompatible Joins (SQL Server PDW)

[image]

Understanding Compatible and Incompatible Joins (SQL Server PDW)
See Also Send Feedback

Describes which SQL Server PDW joins require data movement by explaining which joins are compatible and incompatible. SQL Server PDW can achieve high performance by performing joins in parallel on the Compute nodes and distributions, and then combining the results on the Control node. To improve query performance, tables and queries need to be designed so that, whenever possible, it is not necessary to move data prior to the join.

Contents

  • Background

  • Compatible Joins

  • Incompatible Joins

Background

As part of the shared nothing architecture, every Compute node and distribution must be able to perform its parallel join operations by using local data. Before running parallel joins, SQL Server PDW copies data as necessary among the Compute nodes to ensure that each distribution can perform its portion of the join with local data. Moving data takes time and usually has a negative impact on query performance. Therefore, it is important to design tables and queries to minimize data movement for joins.

Compatible Joins

A compatible join in SQL Server PDW, is a join that does not require data movement before each Compute node or distribution performs its portion of the join. To improve query performance, one design goal is to minimize data movement by using compatible joins when possible.

Join type

Left Table

Right Table

Compatibility

  • All join types

Replicated

Replicated

Compatible - no data movement required.

  • Inner Join

  • Right Outer Join

  • Cross Join

Replicated

Distributed

Compatible – no data movement required.

  • Inner Join

  • Left Outer Join

  • Cross Join

Distributed

Replicated

Compatible – no data movement required.

All join types, except cross joins, can be compatible. See the Compatibility column for details.

Distributed

Distributed

Compatible – no data movement required if the join predicate meets the following conditions:

  • Predicate is an equality join.

  • Predicate joins two distributed columns that have matching data types.

For example, if table A is distributed on column a and table B is distributed on column b, and both a and b have matching data types, the following join is compatible: SELECT * FROM A JOIN B ON(A.a=B.b)

SQL Server PDW analyzes the logic for some conjunctive predicates to see if they are compatible. For example, the following join is compatible: SELECT * FROM A JOIN B ON(A.a=B.b AND A.a = B.b+1)

Cross Joins are always incompatible.

The following example shows the query plan for a compatible inner join between two distributed tables, FactSurveyResponse and DimCustomer. Both tables have CustomerKey( int, NOT NULL ) as the distribution column, and the join is performed on the CustomerKey column.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
EXPLAIN
SELECT *
FROM dbo.FactSurveyResponse AS s JOIN dbo.DimCustomer AS c
    ON (s.CustomerKey = c.CustomerKey);

[TOP]

Incompatible Joins

An incompatible join is a join that requires data movement before each Compute node or distribution can perform its portion of the parallel join. SQL Server PDW automates the data movement by incorporating the data movement operations into the query plan.

Data movement operations take extra time and storage, and can negatively impact query performance. Therefore, one of the design goals for SQL Server PDW databases and queries is to minimize data movement whenever possible by using compatible joins, and by avoiding incompatible joins.

Join type

Left Table

Right Table

Compatibility

  • Left Outer Join

  • Full Outer Join

Replicated

Distributed

Incompatible – requires data movement before the join.

  • Right Outer Join

  • Full Outer Join

Distributed

Replicated

Incompatible – requires data movement before the join.

See Compatibility column for details.

Distributed

Distributed

Incompatible – requires data movement unless the join predicate meets the following conditions:

  • Predicate is an equality join.

  • Predicate joins two distributed columns that have matching data types.

Cross joins are always incompatible.

The following example shows the query plan for an incompatible join between two distributed tables, DimCustomer and ProspectiveBuyer. The join is performed on BirthDate, which is not a distribution column in either table. Thus, a SHUFFLE_MOVE is required.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
EXPLAIN
SELECT c.BirthDate, c.CustomerKey, c.FirstName, c.LastName, p.ProspectiveBuyerKey, p.FirstName, p.LastName
FROM dbo.DimCustomer AS c JOIN dbo.ProspectiveBuyer AS p
    ON (c.BirthDate = p.BirthDate)
where c.LastName = p.LastName
order by c.BirthDate;

In the resulting query plan, there is a SHUFFLE_MOVE operation which indicates that data movement is required before each distribution can run the join.

[TOP]

See Also


Running Prepared Statements (SQL Server PDW)

[image]

Running Prepared Statements (SQL Server PDW)
See Also Send Feedback

This topic describes how to use prepared statements to submit queries to SQL Server PDW. Prepared statements, also called parameterized statements, are SQL statements that include one or more parameters in place of embedded values. Prepared statements are used by business intelligence tools and other applications that run the same query using different values in place of the parameters each time the query is submitted.

For more information about data access, see the Data Developer Center on MSDN.

Caution note Caution

SET FMTONLY can only be used with SQL Server Native Client. The use of SET FMTONLY to return metadata will be removed in the next version of SQL Server PDW. Do not use this feature in new development work. Modify applications that currently use this feature as soon as possible. Use the driver APIs to retrieve metadata instead.

Contents

  • Support for Named Query Parameters

  • Support for Unnamed Query Parameters (?)

  • Limitations and Restrictions

  • Code Samples

Support for Named Query Parameters

Named parameters are supported only for connections that use SQL Server Native Client. Named parameters cannot be used with connections that use Client Tools for SQL Server PDW.

Support for Unnamed Query Parameters (?)

SQL Server PDW supports unnamed parameters for SQL Server Native Client connections, according to the supported scenarios listed in the following table.

All unnamed parameters use the ‘?’ syntax.

Additionally, any of the supported SELECT use cases can be used in the select portion of a CREATE TABLE AS SELECT, the CREATE REMOTE TABLE statement, or as a subquery of another command type that supports prepared statements.

Statement | Supported Locations | Supported Syntaxes
SELECT | JOIN, WHERE, IN, LIKE, and in subqueries | SELECT * FROM tbl1 JOIN tbl2 ON tbl1.col1 = tbl2.col1 AND tbl1.col2 = ?; SELECT * FROM tbl1 WHERE col1= ? AND col2 <= ?; SELECT col2 FROM tbl1 WHERE col1 IN (?); SELECT col2 FROM tbl1 WHERE col1 LIKE (?); SELECT * FROM tbl1 WHERE col2 = (SELECT min(col1) FROM tbl2 WHERE col2= ?);
INSERT | VALUES and in subqueries | INSERT INTO tbl1 VALUES (?); INSERT INTO tbl1 VALUES (?,?); INSERT INTO tbl1 VALUES (SELECT col1 FROM tbl1 WHERE col1 = ?));
UPDATE | SET, WHERE, IN, LIKE, and in subqueries | UPDATE tbl1 SET col1 = ?, col2 = ? WHERE col2 > ?; UPDATE tbl1 SET col1 = 0 WHERE col1 IN (?); UPDATE tbl1 SET col2 = 2 WHERE col1 LIKE ?; UPDATE tbl1 SET col1 = ?, col2 = (SELECT min(col1) FROM tbl2 WHERE col2= ?);
DELETE | WHERE, IN, LIKE, and in subqueries | DELETE FROM tbl1 WHERE col1 = ?; DELETE FROM tbl1 WHERE col2 IN (?); DELETE FROM tbl1 WHERE col1 LIKE ?; DELETE FROM tbl1 WHERE col1 = (SELECT count(*) FROM tbl2 WHERE col1 = ?);
DECLARE | Assignment | DECLARE @p1 int = ?; DECLARE @p1 int = (SELECT col1 FROM tbl1 WHERE col1 =?);
SET | Assignment and in subqueries | SET @p1 = ?; SET @p1 = 'SELECT * FROM tbl1 WHERE col2 = (SELECT min(col1) FROM tbl2 WHERE col2= ?)';

[TOP]

Limitations and Restrictions

Money and smallmoney data types are not natively supported in the ADO.NET, OLE DB, and ODBC protocol for SQL Server PDW. To pass a currency value to a parameter in a prepared statement, you can represent money or smallmoney as a parameter of type decimal.

If a prepared statement inserts a value of type decimal into an integer data type, the operation will succeed and the portion of the data to the right of the decimal point will be truncated. For example, when a prepared statement inserts $1234.5678 into a SQL Server PDW smallint, the resulting smallint is 1234.

Following is a list of specific limitations to SQL Server PDW prepared statement support. This list is not exhaustive.

Restriction | Invalid Syntax Examples
SQL Server PDW does not support the reflexive property in the context of parameterization. Parameterized statements always require the column information on the left side of an expression. | SELECT col1 FROM tbl1 WHERE ? = col1;
Parameters cannot be specified in the select list of a SELECT statement. | SELECT col1, ? FROM tbl1 WHERE col1 = 4;
Parameters cannot be specified in the FROM portion of a SELECT statement. | SELECT col1 FROM ? WHERE col1 = 4;
When providing parameters for an IN clause, no more than two values can be used and all values must be parameters. | UPDATE tbl1 SET col1 = 2 WHERE col1 in (?,?,?); SELECT col1, col2 FROM tbl1 WHERE col1 in (?,10);
The use of parameters is not supported with the BETWEEN clause. | DELETE FROM tbl1 WHERE col1 between 1 AND ?;
Parameters cannot be used as part of expressions, including as part of binary operands or computed columns. | SELECT col1 FROM tbl1 WHERE col1 = (1+ ?); SET @p1 = (?+5);

[TOP]

Code Examples

The following table provides sample code for submitting prepared statements using each of the supported connection API’s.

Connection API | Example Code
ADO.NET | DbProviderFactory factory = DbProviderFactories.GetFactory(“Microsoft.SqlServer.DataWarehouse”); DbConnection connection = factor.CreateConnection(); DbCommand command = connection.CreateCommand(“SELECT * FROM bing WHERE id=?”); DbParameter param1 = new DbParameter(); Param1.Value = 1; Command.Parameters.Add(param1); Command.ExecuteQuery(); Command.Close();
ODBC | OdbcCommand command = OdbcConnection.CreateCommand(“SELECT * FROM bing WHERE id=?”); OdbcParameter param1 = new OdbcParameter(); param1.value = 1; command.Parameters.Add(param1); Command.Prepare(); Command.Execute(); Command.Close();
OLEDB | OleDbCommand command = OleDbConnection.CreateCommand(“SELECT * FROM bing WHERE id=?”); OleDbParameter param1 = new OleDbParameter(); param1.value = 1; command.Parameters.Add(param1); Command.Prepare(); Command.Ecxecute(); Command.Close();

[TOP]

See Also


Monitoring Active Queries (SQL Server PDW)

[image]

Monitoring Active Queries (SQL Server PDW)
See Also Send Feedback

This topic shows how to use the Admin Console and the SQL Server PDW system views to monitor active queries. See Monitor the Appliance by Using the Admin Console (Analytics Platform System) and System Views (SQL Server PDW) for information on these tools.

Prerequisites

Regardless of the method used to monitor active queries, the login must have the permissions described in “Use All of the Admin Console” in Grant Permissions to Use the Admin Console (SQL Server PDW).

Monitor Active Queries

Both the Admin Console and the SQL Server PDW system views can be used to monitor active queries. Follow the instructions below.

To monitor active queries by using the Admin Console

  1. Log on to the Admin Console. See Monitor the Appliance by Using the Admin Console (Analytics Platform System) for instructions.

  2. On the top menu, click Queries. You will see a table with basic information about the most recent queries on the appliance, including the login who submitted the query, the start and end times for the query, and the current status of the query.

  3. To see the query command, hover the mouse pointer over the query ID in the left column for that row.

    To see more detailed information for a particular query, click the query ID. You will see information including the full query and the query plan, with status information for each step in the query execution. If any errors were returned, you can also see detailed information on the errors. See Understanding Query Plans (SQL Server PDW) for information on how to interpret the query plan information available in the Admin Console.

To monitor active queries by using the system views

The primary system view used to monitor queries is sys.dm_pdw_exec_requests (SQL Server PDW). Use this system view to find the request_id for an active or recent query, based on the query text.

For example, the following query finds the request_id and the current status for any query that selects all columns from the memberAddresses table.

Copy image Copy Code
SELECT request_id, command, status
FROM sys.dm_pdw_exec_requests
WHERE command
LIKE ‘%SELECT * FROM db1..memberAddresses%’;

After the request_id has been identified for a query, use the other information in the dm_pdw_exec_requests table to find out about the processing of the query, or use sys.dm_pdw_request_steps (SQL Server PDW) to view the status of the individual query steps for the query execution.

See Also


Queries on External Tables With Predicate Pushdown (SQL Server PDW)

[image]

Queries on External Tables With Predicate Pushdown (SQL Server PDW)
Send Feedback

Explanations and examples of using predicate pushdown when querying Hadoop data with a SQL Server PDW query. Use this to improve performance of queries on external tables by performing some of the query work on Hadoop in order to reduce the size of the data that must be copied to SQL Server PDW.

Examples

All of these queries assume that the job tracker ID was defined for the Hadoop data source.

Predicate pushdown for selecting a subset of rows

You can use predicate pushdown to improve performance for a query that selects a subset of rows from an external table.

In this query, SQL Server PDW will initiate a map-reduce job to retrieve the rows that match the predicate customer.account_balance < 200000 on Hadoop. Since the query can complete successfully without scanning all of the rows in the table, only the rows that meet the predicate criteria are copied to SQL Server PDW. This saves significant time and requires less temporary storage space when the number of customer balances < 200000 is small in comparison with the number of customers with account balances >= 200000.

Copy image Copy Code
SELECT * FROM customer WHERE customer.account_balance < 200000

Predicate pushdown for selecting a subset of columns

You can use predicate pushdown to improve performance for a query that selects a subset of columns from an external table.

In this query, SQL Server PDW will initiate a map-reduce job to pre-process the Hadoop delimited-text file so that only the data for the two columns, customer.name and customer.zip_code, will be copied to SQL Server PDW.

Copy image Copy Code
SELECT customer.name, customer.zip_code FROM customer WHERE customer.account_balance < 200000

Predicate pushdown for basic expressions and operators

SQL Server PDW will consider the following basic expressions and operators for predicate pushdown.

  • Binary comparison operators ( <, >, =, !=, <>, >=, <= ) for numeric, date, and time values.

  • Arithmetic operators ( +, -, *, /, % ).

  • Logical operators (AND, OR).

  • Unary operators (NOT, IS NULL, IS NOT NULL).

The operators BETWEEN, NOT, IN, and LIKE might be pushed-down. This depends on how the query optimizer rewrites them as a series of statements that use basic relational operators.

The following query has multiple predicates that can be pushed down to Hadoop. SQL Server PDW can push map-reduce jobs to Hadoop to perform the predicate customer.account_balance <= 200000. The expression BETWEEN 92656 and 92677 is comprised of binary and logical operations that can be pushed to Hadoop. The logical AND of customer.account_balance and customer.zipcode is a final expression.

Putting this altogether, the map-reduce jobs can perform all of the WHERE clause. Only the data that meets the SELECT criteria will be copied back to SQL Server PDW.

Copy image Copy Code
SELECT * FROM customer WHERE customer.account_balance <= 200000 AND customer.zipcode BETWEEN 92656 AND 92677


Load (SQL Server PDW)

[image]

Load (SQL Server PDW)
See Also Send Feedback

You can load or insert data into SQL Server PDW by using Integration Services, the dwloader command-line tool, or the SQL INSERT statement.

Data Loading Basics

Loading Environment

To load data, you need one or more loading servers. You can use your own existing ETL or other servers, or you can purchase new servers. For more information, see Acquire and Configure a Loading Server (SQL Server PDW). These instructions include a Loading Server Capacity Planning Worksheet (SQL Server PDW) to help you plan the right solution for loading.

Loading with dwloader

Using the dwloader Command-Line Loader (SQL Server PDW) is the fastest way to load data into SQL Server PDW.

dwloader loads data directly to the Compute nodes without passing the data through the Control node. To load data, dwloader first communicates with the Control node to obtain contact information for the Compute nodes. dwloader sets up a communication channel with each Compute node and then sends 256KB chunks of data to the Compute nodes in a round-robin manner.

On each Compute node, Data Movement Service (DMS) receives and processes the chunks of data. Processing the data includes converting each row into SQL Server native format, and computing the distribution hash to determine the Compute node to which each row belongs.

After processing the rows, DMS uses a shuffle move to transfer each row to the correct Compute node and instance of SQL Server. As SQL Server receives the rows, it batches them according to the –b batch size parameter set in dwloader, and then bulk loads the batch.

Related Tasks

Task | Description
Create a staging database. | Create the Staging Database (SQL Server PDW)
Load data with Integration Services. | Load Data With Integration Services (SQL Server PDW)
Understand type conversions for dwloader. | Data Type Conversion Rules for dwloader (SQL Server PDW)
Load data with dwloader. | dwloader Command-Line Loader (SQL Server PDW)
Understand type conversions for INSERT. | Load Data With INSERT (SQL Server PDW)

See Also

Concepts

Grant Permissions to Load Data (SQL Server PDW)


Create the Staging Database (SQL Server PDW)

[image]

Create the Staging Database (SQL Server PDW)
See Also Send Feedback

SQL Server PDW uses a staging database to store data temporarily during the load process. By default, SQL Server PDW uses the destination database as the staging database which can cause table fragmentation. To reduce table fragmentation, you can create a user-defined staging database. Or, when rollback from a load failure is not a concern, you can use the fastappend loading mode to improve performance by skipping the temporary table and loading directly into the destination table.

Contents

  • Staging Database Basics

  • Permissions

  • Best Practices: Creating a Staging Database

  • Examples

Staging Database Basics

A staging database is a user-created SQL Server PDW database that stores data temporarily while it is being loaded into the appliance. When a staging database is specified for a load, the appliance first copies the data to the staging database and then copies the data from temporary tables in the staging database to permanent tables in the destination database.

When a staging database is not specified for a load, SQL Server PDW creates the temporary tables in the destination database and uses them to store the loaded data before it inserts the loaded data into the permanent destination tables.

When a load uses the fastappend mode, SQL Server PDW skips using temporary tables altogether and appends the data directly to the target table. The fastappend mode improves load performance for ELT scenarios where data is loaded into a table that is a temporary table from the application standpoint. For example, an ELT process could load data into a temporary table, process the data by cleansing and de-duping, and then insert the data into the target fact table. In this case, it is not necessary for SQL Server PDW to first load the data into a SQL Server PDW internal temporary table before inserting the data into the application’s temporary table. The fastappend mode avoids the extra load step, which significantly improves the load performance. Note that to use the fastappend mode, you must use multi-transaction mode, which means that recovery from a failed or aborted load must be handled by your own load process.

Staging Database Benefits

The primary benefit of a staging database is to reduce table fragmentation. If a staging database is not used, the data is loaded into temporary tables in the destination database. When temporary tables get created and dropped in the destination database, the pages for the temporary tables and permanent tables become interleaved. Over time, this causes table fragmentation and degrades performance. In contrast, a staging database ensures that temporary tables are created and dropped in a separate file space than the permanent tables.

Staging Database Table Structures

The storage structure for each database table depends on the destination table.

  • For loads into a heap or clustered columnstore index, the staging table is a heap.

  • For loads into a rowstore clustered index, the staging table is a rowstore clustered index.

[TOP]

Permissions

Requires CREATE permission (for creating a temporary table) on the staging database. For more information, see Grant Permissions to Load Data (SQL Server PDW).

[TOP]

Best Practices: Creating a Staging Database

  1. There should only be one staging database per appliance. This can be shared by all load jobs for all destination databases.

  2. The size of the staging database is customer-specific. Initially, when first populating the appliance, the staging database should be large enough to accommodate the initial load jobs. These load jobs tend to be large because multiple loads can occur concurrently. After initial load jobs have completed and the system is in production, the size of each load job is likely to be smaller. When this occurs, you can reduce the size of the staging database to accommodate the smaller load sizes. To reduce the size, you can drop the staging database and create it again with smaller size allocations, or you can use the ALTER DATABASE (SQL Server PDW) statement.

    When creating the staging database, use the following guidelines.

    • The replicated table size should be the estimated size, per Compute node, of all the replicated tables that will load concurrently. This is typically 25-30 GB.

    • The distributed table size should be the estimated size, per appliance, of all the distributed tables that will load concurrently.

    • The log size is typically similar to the replicated table size.

[TOP]

Examples

A. Creating a Staging Database

The following example creates a staging database, Stagedb, for use with all loads on the appliance. Suppose you estimate that five replicated tables of size 5 GB each will load concurrently. This results in allocating at least 25 GB for the replicated size. Suppose you estimate that six distributed tables of sizes 100, 200, 400, 500, 500, and 550 GB will load concurrently. This results in allocating at least 2250 GB for the distributed table size.

Copy image Copy Code
CREATE DATABASE Stagedb
WITH (

    AUTOGROW = ON,

    REPLICATED_SIZE = 25 GB,

    DISTRIBUTED_SIZE = 2250 GB,

    LOG_SIZE = 25 GB

);

[TOP]

See Also


Load Data With Integration Services (SQL Server PDW)

[image]

Load Data With Integration Services (SQL Server PDW)
See Also Send Feedback

Provides reference and deployment information for loading data into SQL Server PDW by using Integration Services packages.

Contents

  • Before You Begin

  • Basics

  • Deploy an Integration Services Package

  • Data Types

  • General Remarks

  • Limitations and Restrictions

  • Locking Behavior

  • Examples

Before You Begin

Before you can start loading data, use the following topics to install the Integration Services Destination Adapters and to create an Integration Services package that connects to SQL Server PDW:

  • Install Integration Services Destination Adapters (SQL Server PDW)

  • Connect With SQL Server Integration Services for Loading (SQL Server PDW)

For general information about developing Integration Services packages, see Designing and Implementing Packages (Integration Services) on MSDN.

Basics

Integration Services is the component of SQL Server for high-performance extraction, transformation, and loading (ETL) of data, and is commonly used to populate and update a data warehouse.

The SQL Server PDW Destination Adapter is an Integration Services component that lets you load data into SQL Server PDW by using Integration Services dtsx packages. In a package workflow for SQL Server PDW, you can load and merge data from multiple sources and load data to multiple destinations. The loads occur in parallel, both within a package and among multiple packages running concurrently, up to a maximum of 10 loads running in parallel on the same appliance.

In addition to the tasks described in this topic, you can use other features of Integration Services to filter, transform, analyze, and cleanse your data before loading it into the data warehouse. You can also enhance the workflow of the package by running SQL statements, running child packages, or sending mail.

For complete documentation of Integration Services, see SQL Server Integration Services on MSDN.

Methods for Running an Integration Services Package

Use one of these methods to run an Integration Services package.

Run from SQL Server 2008 R2 Business Intelligence Development Studio (BIDS)

To run the package from within BIDS, right-click on your package and choose Execute Package.

By default, BIDS runs packages using 64-bit binaries. This is determined by the Run64BitRuntime package property. To set this property, go to Solution Explorer, right-click on your project and choose Properties. On the Integration Services Property Pages, go to Configuration Properties and select Debugging. You will see the Run64BitRuntime property under the Debug Options. To use 32-bit runtimes, set this to False. To use 64-bit runtimes, set this to True.

Run from SQL Server 2012 SQL Server Data Tools

To run the package from within SQL Server Data Tools, right-click on your package and choose Execute Package.

Run From PowerShell

To run the package from Windows PowerShell, using the dtexec utility: dtexec /FILE <packagePath>

For example, dtexec /FILE "C:\Users\User1\Desktop\Package.dtsx"

Run From a Windows Command Prompt

To run the package from a Windows command prompt, using the dtexec utility: dtexec /FILE <packagePath>

For example: dtexec /FILE "C:\Users\User1\Desktop\Package.dtsx"

Data Types

When using Integration Services to load data from a data source to a SQL Server PDW database, the data is first mapped from the source data to Integration Services data types. This allows data from multiple data sources to map to a common set of data types.

Then the data is mapped from Integration Services to SQL Server PDW data types. For each SQL Server PDW data type, the following table lists the Integration Services data types that can be converted to the SQL Server PDW data type.

SQL Server PDW Data Type | Integration Services Data Type(s) That Map to the SQL Server PDW Data Type
BIT | DT_BOOL
BIGINT | DT_I1, DT_I2, DT_I4, DT_I8, DT_UI1, DT_UI2, DT_UI4
CHAR | DT_STR
DATE | DT_DBDATE
DATETIME | DT_DATE, DT_DBDATE, DT_DBTIMESTAMP, DT_DBTIMESTAMP2
DATETIME2 | DT_DATE, DT_DBDATE, DT_DBTIMESTAMP, DT_DBTIMESTAMP2
DATETIMEOFFSET | DT_WSTR
DECIMAL | DT_DECIMAL, DT_I1, DT_I2, DT_I4, DT_I4, DT_I8, DT_NUMERIC, DT_UI1, DT_UI2, DT_UI4, DT_UI8
FLOAT | DT_R4, DT_R8
INT | DT_I1, DTI2, DT_I4, DT_UI1, DT_UI2
MONEY | DT_CY
NCHAR | DT_WSTR
NUMERIC | DT_DECIMAL, DT_I1, DT_I2, DT_I4, DT_I8, DT_NUMERIC, DT_UI1, DT_UI2, DT_UI4, DT_UI8
NVARCHAR | DT_WSTR, DT_STR
REAL | DT_R4
SMALLDATETIME | DT_DBTIMESTAMP2
SMALLINT | DT_I1, DT_I2, DT_UI1
SMALLMONEY | DT_R4
TIME | DT_WSTR
TINYINT | DT_I1
VARBINARY | DT_BYTES
VARCHAR | DT_STR

Limited Support for Data Type Precision

SQL Server PDW generates a validation error if you map a DT_NUMERIC or DT_DECIMAL input column that contains a value with precision greater than 28.

Unsupported Data Types

SQL Server PDW does not support the following Integration Services data types:

  • DT_DBTIMESTAMPOFFSET

  • DT_DBTIME2

  • DT_GUID

  • DT_IMAGE

  • DT_NTEXT

  • DT_TEXT

To load columns that contain data of these types into SQL Server PDW, you must add a Data Conversion transformation upstream in the data flow to convert the data to a compatible data type.

[TOP]

Permissions

To run an Integration Services load package, you need:

  • LOAD permission on the database.

  • Applicable INSERT, UPDATE, DELETE permissions on the destination table.

  • If a staging database is used, CREATE permission on the staging database. This is for creating a temporary table.

  • If no staging database is used, CREATE permission on the destination database. This is for creating a temporary table.

[TOP]

General Remarks

When an Integration Services package has multiple SQL Server PDW destinations running and one of the connections is terminated, Integration Services stops pushing data to all of the SQL Server PDW destinations.

[TOP]

Limitations and Restrictions

For an Integration Services package, the number of SQL Server PDW destinations for the same data source is limited by the maximum number of active loads. The maximum is pre-configured and is not user-configurable. For the maximum number of loads and queued loads per appliance, see Minimum and Maximum Values (SQL Server PDW).

Each Integration Services package destination for the same data source counts as one load when the package is running. For example, suppose the maximum active loads is 10. The package will not run if it attempts to open 11 or more destinations for the same data source.

Multiple packages can run concurrently as long as each package does not use more than the maximum active loads. For example, if the maximum active loads is 10, you can concurrently run two packages that each use 10 destinations. One package will run while the other one waits in the load queue.

If the number of loads in the load queue exceeds the maximum queued loads, the package will not run. For example, if the maximum number of loads is 10 per appliance and the maximum number of queued loads is 40 per appliance, you can concurrently run five Integration Services packages that each open 10 destinations. If you try to run a sixth package, it will not run.

[TOP]

Locking Behavior

When loading data with Integration Services, SQL Server PDW uses row-level locks to update data in the destination table. This means that each row is locked for read and write while it is being updated. The rows in the destination table are not locked while the data is loaded into the staging table.

[TOP]

Examples

A. Simple Flat File Data Load

The following walkthrough demonstrates a simple data load using Integration Services to load flat file data to a SQL Server PDW appliance. This example assumes that Integration Services has already been installed on the client machine, and the SQL Server PDW destination has been installed, as described above.

In this example we will load into the Orders table, which has the following DDL. The Orders table is part of the LoadExampleDB database.

Copy image Copy Code
CREATE TABLE LoadExampleDB.dbo.Orders (
   id INT,
   city varchar(25),
   lastUpdateDate DATE,
   orderDate DATE)
;

Here is the load data:

Copy image Copy Code
id        city           lastUpdateDate     orderdate
--------- -------------- ------------------ ----------
1         Seattle        2010-05-01         2010-01-01
2         Denver         2002-06-25         1999-01-02

In preparation for the load, create the flat file exampleLoad.txt, containing the load data:

Copy image Copy Code
id,city,lastUpdateDate,orderDate
1,Seattle,2010-05-01,2010-01-01
2,Denver,2002-06-25,1999-01-02

First, create an Integration Services package by performing these steps:

  1. In SQL Server Data Tools (SSDT), select File, New, and then Project. Select Integration Services Project from the options listed. Name this project ExampleLoad, and click OK.

  2. Click the Control Flow tab and then drag the Data Flow Task from the Toolbox to the Control Flow pane.

  3. Click the Data Flow tab and then drag Flat File Source from the Toolbox to the Data Flow pane. Double-click the box you just created to open the Flat File Source Editor.

  4. Click Connection Manager and then click New.

  5. In the Connection manager name box, enter a friendly name for the connection. For this example, Example Load Flat File CM.

  6. Click Browse and select the ExampleLoad.txt file from the local machine.

  7. Since the flat file contains a row with column names, click the Column names in the first data row box.

  8. Click Columns in the left column, and preview the data that will be loaded to make sure the column names and data were interpreted correctly.

  9. Click Advanced in the left column. Click on each column name to review the data type that has been associated with the data. Type changes in the box so that the data types of the loaded data will be compatible with the destination column types.

  10. Click OK to save your connection manager.

  11. Click OK to exit the Flat File Source Editor.

Specify the destination for the data flow.

  1. Drag the SQL Server PDW Destination from the Toolbox to the Data Flow pane.

  2. Double-click the box you just created to load the SQL Server PDW Destination Editor.

  3. Click the down arrow next to Connection Manager.

  4. Select Create a New Connection.

  5. Fill in the information for the server, user, password, and destination database with information specific to your appliance. (Examples are shown below). Then click OK.

    For InfiniBand connections, Server name: Enter <appliance-name>-SQLCTL01,17001.

    For Ethernet connections, Server name: Enter the IP address of the Control node cluster, comma, port 17001. For example, 10.192.63.134,17001.

    User: user1

    Password: password1

    Destination Database: LoadExampleDB

  6. Select the destination table: Orders.

  7. Select Append as the loading mode and click OK.

Specify the data flow from source to destination.

  1. On the Data Flow pane, drag the green arrow from the Flat File Source box to the SQL Server PDW Destination box.

  2. Double-click the SQL Server PDW Destination box so that you see the SQL Server PDW Destination Editor again. You should see the column names from the flat file on the left, under Unmapped Input Columns. You should see the column names from the destination table on the right, under Unmapped Destination Columns. Map the columns by dragging or double-clicking the matching column names in the Unmapped Input Columns and Unmapped Destination Columns lists to the Mapped Columns box. Click OK to save your settings.

  3. Save the package by clicking Save in the File menu.

Run the package on your computer Integration Services.

  1. In the Integration Services Solution Explorer (right column), right-click Package.dtsx and select Execute.

  2. The package will run and the progress plus any errors will be shown on the Progress pane. Use a SQL client to confirm the load, or monitor the load via the SQL Server PDW Admin Console.

[TOP]

See Also

Concepts

How to create a script task that uses the SSIS PDW destination adapter (SQL Server PDW)
Grant Permissions to Load Data (SQL Server PDW)
Common Metadata Query Examples (SQL Server PDW)

Other Resources

SQL Server Integration Services
Designing and Implementing Packages (Integration Services)
Tutorial: Creating a Basic Package Using a Wizard
Getting Started (Integration Services)
Dynamic Package Generation Sample
Designing Your SSIS Packages for Parallelism (SQL Server Video)
Microsoft SQL Server Community Examples: Integration Services
Improving Incremental Loads with Change Data Capture
Slowly Changing Dimension Transformation
Bulk Insert Task


How to create a script task that uses the SSIS PDW destination adapter (SQL Server PDW)

[image]

How to create a script task that uses the SSIS PDW destination adapter (SQL Server PDW)
See Also Send Feedback

The Integration Services Script Task allows you to perform virtually any operation that can be accomplished in a .Net application within the context of an SSIS control flow. This SSIS package script code is a code example for using the SSIS PDW destination adaptor.

Sample Code

  1. Create a blank Script Task in SSIS.

  2. Open the task in the Script Task Editor, and click Edit Script.

  3. Edit the script to use the code provided below. Customized the script to your environment by making the following changes:

    • You must retain the namespace of the script created by the script task. Note the name of the namespace namespace ST_<GUID> and edit the script below, changing ST_<GUID> to the original GUID of the script task.

    • Replace four occurrences of the placeholder value XXXXXXXX with the values appropriate to your environment and intended action.

    C# Copy image Copy Code
    #region Help:  Introduction to the script task
/* The Script Task allows you to perform virtually any operation that can be accomplished in
 * a .Net application within the context of an Integration Services control flow.
 *
 * Expand the other regions which have "Help" prefixes for examples of specific ways to use
 * Integration Services features within this script task. */
#endregion

#region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using Microsoft.SqlServer.Management.IntegrationServices;
using Microsoft.SqlServer.Dts.Pipeline;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using System.Runtime.InteropServices;
using System.Reflection;
#endregion

namespace ST_<GUID>
{
    /// <summary>
    /// ScriptMain is the entry point class of the script.  Do not change the name, attributes,
    /// or parent of this class.
    /// </summary>
[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
        #region Help:  Using Integration Services variables and parameters in a script
        /* To use a variable in this script, first ensure that the variable has been added to
         * either the list contained in the ReadOnlyVariables property or the list contained in
         * the ReadWriteVariables property of this script task, according to whether or not your
         * code needs to write to the variable.  To add the variable, save this script, close this instance of
         * Visual Studio, and update the ReadOnlyVariables and
         * ReadWriteVariables properties in the Script Transformation Editor window.
         * To use a parameter in this script, follow the same steps. Parameters are always read-only.
         *
         * Example of reading from a variable:
         *  DateTime startTime = (DateTime) Dts.Variables["System::StartTime"].Value;
         *
         * Example of writing to a variable:
         *  Dts.Variables["User::myStringVariable"].Value = "new value";
         *
         * Example of reading from a package parameter:
         *  int batchId = (int) Dts.Variables["$Package::batchId"].Value;
         *
         * Example of reading from a project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].Value;
         *
         * Example of reading from a sensitive project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].GetSensitiveValue();
         * */

        #endregion

        #region Help:  Firing Integration Services events from a script
        /* This script task can fire events for logging purposes.
         *
         * Example of firing an error event:
         *  Dts.Events.FireError(18, "Process Values", "Bad value", "", 0);
         *
         * Example of firing an information event:
         *  Dts.Events.FireInformation(3, "Process Values", "Processing has started", "", 0, ref fireAgain)
         *
         * Example of firing a warning event:
         *  Dts.Events.FireWarning(14, "Process Values", "No values received for input", "", 0);
         * */
        #endregion

        #region Help:  Using Integration Services connection managers in a script
        /* Some types of connection managers can be used in this script task.  See the topic
         * "Working with Connection Managers Programatically" for details.
         *
         * Example of using an ADO.Net connection manager:
         *  object rawConnection = Dts.Connections["Sales DB"].AcquireConnection(Dts.Transaction);
         *  SqlConnection myADONETConnection = (SqlConnection)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Sales DB"].ReleaseConnection(rawConnection);
         *
         * Example of using a File connection manager
         *  object rawConnection = Dts.Connections["Prices.zip"].AcquireConnection(Dts.Transaction);
         *  string filePath = (string)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Prices.zip"].ReleaseConnection(rawConnection);
         * */
        #endregion

        public static string GetSymbolicName(int errorCode)
        {
            string symbolicName = string.Empty;
            HResults hresults = new HResults();

            foreach (FieldInfo fieldInfo in hresults.GetType().GetFields())
            {
                if ((int)fieldInfo.GetValue(hresults) == errorCode)
                {
                    symbolicName = fieldInfo.Name;
                    break;
                }
            }

            return symbolicName;
        }
     /// <summary>
        /// This method is called when this script task executes in the control flow.
        /// Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
        /// To open Help, press F1.
        /// </summary>
        public void Main()
        {
            try
            {
                Package package = new Package();
                Executable dataFlowtask = package.Executables.Add("STOCK:PipelineTask");

                TaskHost taskHost = dataFlowtask as TaskHost;
                taskHost.Name = "Data Flow Task";

                MainPipe pipeline = taskHost.InnerObject as MainPipe;

                //Connection to the Oracle Source created
                ConnectionManager connectionSource = package.Connections.Add("OLEDB");
                connectionSource.Name = "OracleSource";
                connectionSource.ConnectionString = "Data Source=127.0.0.1;User ID=sa;Password=XXXXXXXX;Provider=OraOLEDB.Oracle.1;Persist Security Info=True;";

                IDTSComponentMetaData100 srcComponent = pipeline.ComponentMetaDataCollection.New();
                srcComponent.ComponentClassID = "DTSAdapter.OleDbSource";
                srcComponent.ValidateExternalMetadata = true;
                IDTSDesigntimeComponent100 srcDesignTimeComponent = srcComponent.Instantiate();
                srcDesignTimeComponent.ProvideComponentProperties();
                srcComponent.Name = "OleDb Source";

                srcDesignTimeComponent.SetComponentProperty("AccessMode", 0);
                srcDesignTimeComponent.SetComponentProperty("OpenRowset", "XXXXXXXX");

                // Set the connection manager
                srcComponent.RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.GetExtendedInterface(connectionSource);
                srcComponent.RuntimeConnectionCollection[0].ConnectionManagerID = connectionSource.ID;

                // Retrieve the column metadata
                srcDesignTimeComponent.AcquireConnections(null);
                srcDesignTimeComponent.ReinitializeMetaData();
                srcDesignTimeComponent.ReleaseConnections();

                ConnectionManager connectionDest = package.Connections.Add("SQLPDW");
                connectionDest.Name = "SQL Server Destination";
                connectionDest.ConnectionString = @"Data Source=127.0.0.1,17001;Initial Catalog=SomeDB;User ID=sa;Password=XXXXXXXX;";

                IDTSComponentMetaData100 destComponent = pipeline.ComponentMetaDataCollection.New();
                destComponent.ComponentClassID = typeof(Microsoft.SqlServer.Dts.Pipeline.SQLPDWDestinationAdapter).AssemblyQualifiedName;
                IDTSDesigntimeComponent100 destDesignTimeComponent = destComponent.Instantiate();
                destDesignTimeComponent.ProvideComponentProperties();

                destComponent.Name = "ADO Destination";
                destDesignTimeComponent.SetComponentProperty("Mode", "Append");
                destDesignTimeComponent.SetComponentProperty("FinalTable", "XXXXXXXX");

                // set connection
                destComponent.RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.GetExtendedInterface(connectionDest);
                destComponent.RuntimeConnectionCollection[0].ConnectionManagerID = connectionDest.ID;

                destDesignTimeComponent.AcquireConnections(null);

                // get metadata
                destDesignTimeComponent.ReinitializeMetaData();
                destDesignTimeComponent.ReleaseConnections();

                //Connect Source and Destination
                IDTSPath100 path = pipeline.PathCollection.New();
                path.AttachPathAndPropagateNotifications(srcComponent.OutputCollection[0], destComponent.InputCollection[0]);

                // Configure the destination
                IDTSInput100 destInput = destComponent.InputCollection[0];
                IDTSVirtualInput100 destVirInput = destInput.GetVirtualInput();
                IDTSInputColumnCollection100 destInputCols = destInput.InputColumnCollection;
                IDTSExternalMetadataColumnCollection100 destExtCols = destInput.ExternalMetadataColumnCollection;
                IDTSOutputColumnCollection100 sourceColumns = srcComponent.OutputCollection[0].OutputColumnCollection;

                // The OLEDB destination requires you to hook up the external columns
                foreach (IDTSOutputColumn100 outputCol in sourceColumns)
                {
                    // Get the external column id
                    IDTSExternalMetadataColumn100 extCol = (IDTSExternalMetadataColumn100)destExtCols[outputCol.Name];
                    if (extCol != null)
                    {
                        // Create an input column from an output col of previous component.
                        destVirInput.SetUsageType(outputCol.ID, DTSUsageType.UT_READONLY);
                        IDTSInputColumn100 inputCol = destInputCols.GetInputColumnByLineageID(outputCol.ID);
                        if (inputCol != null)
                        {
                            // map the input column with an external metadata column
                            destDesignTimeComponent.MapInputColumn(destInput.ID, inputCol.ID, extCol.ID);
                        }
                    }
                }

                string pkg = @"D:\Test\GenPackage.dtsx";
                Microsoft.SqlServer.Dts.Runtime.Application app = new Microsoft.SqlServer.Dts.Runtime.Application();
                app.SaveToXml(pkg, package, null);

                // Now execute the package after creating it.
                package.Execute();
            }
            catch (COMException ex)
            {
                string symbolicName = GetSymbolicName(ex.ErrorCode);
                Console.WriteLine("Symbolic Name: {0}", symbolicName);
                string zz = symbolicName;
            }

            Dts.TaskResult = (int)ScriptResults.Success;
        }

        #region ScriptResults declaration
        /// <summary>
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        ///
        /// This code was generated automatically.
        /// </summary>
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

     }
}

See Also

Concepts

Load Data With Integration Services (SQL Server PDW)
Install Integration Services Destination Adapters (SQL Server PDW)


Data Type Conversion Rules for dwloader (SQL Server PDW)

[image]

Data Type Conversion Rules for dwloader (SQL Server PDW)
See Also Send Feedback

This topic describes the input data formats and implicit data type conversions that dwloader Command-Line Loader (SQL Server PDW) supports when it loads data into SQL Server PDW. The implicit data conversions occur when the input data does not match the data type in the SQL Server PDW target table. Use this information when designing your loading process to ensure your data will load successfully into SQL Server PDW.

For more information about the loader, see. dwloader Command-Line Loader (SQL Server PDW).

Contents

  • Inserting Literals into Binary Types

  • Inserting Literals into Date and Time Types

  • Inserting Literals into Numeric Types

  • Inserting Literals into String Types

Inserting Literals into Binary Types

The following table defines the accepted literal types, format, and conversion rules for loading a literal value into a SQL Server PDW column of type binary (n) or varbinary(n).

Input Data Type | Input Data Examples | Conversion to binary or varbinary Data Type
Binary literal | [0x]hexidecimal_string Example: 12Ef or 0x12Ef | The 0x prefix is optional. The data source length cannot exceed the number of bytes specified for the data type. If the data source length is less than size of the binary data type, the data is padded to the right with zeros to reach the data type size.

[TOP]

Inserting Literals into Date and Time Types

Date and time literals are represented by using string literals in specific formats, enclosed in single quotation marks. The following tables define the allowed literal types, format, and conversion rules for loading a date or time literal into a column of type datetime, smalldatetime, date, time, datetimeoffset, or datetime2. The tables define the default format for the given data type. Other formats that can be specified are defined in the section Datetime Formats. Date and time literals cannot include leading or trailing spaces. date, smalldatetime, and null values cannot be loaded in fixed width mode.

datetime Data Type

The following table defines the default format and rules for loading literal values into a column of type datetime. An empty string ('') is converted to the default value '1900-01-01 12:00:00.000'. Strings that contain only blanks (' ') generate an error.

Input Data Type | Input Data Examples | Conversion to datetime Data Type
String literal in datetime format | 'yyyy-MM-dd hh:mm:ss[.fff]' Example: '2007-05-08 12:35:29.123' | Missing fractional digits are set to 0 when the value is inserted. For example, the literal '2007-05-08 12:35' is inserted as '2007-05-08 12:35:00.000'.
String literal in smalldatetime format | 'yyyy-MM-dd hh:mm' Example: '2007-05-08 12:35' | Seconds and remaining fractional digits are set to 0 when the value is inserted.
String literal in date format | 'yyyy-MM-dd' Example: '2007-05-08' | Time values (hour, minutes, seconds, and fractions) are set to 12:00:00.000 when the value is inserted.
String literal in datetime2 format | 'yyyy-MM-dd hh:mm:ss.fffffff' Example: '2007-05-08 12:35:29.1234567' | The source data cannot exceed three fractional digits. For example, the literal '2007-05-08 12:35:29.123' will be inserted, but the value '2007-05-8 12:35:29.1234567' generates an error.

smalldatetime Data Type

The following table defines the default format and rules for loading literal values into a column of type smalldatetime. An empty string ('') is converted to the default value '1900-01-01 12:00'. Strings that contain only blanks (' ') generate an error.

Input Data Type

Input Data Examples

Conversion to smalldatetime Data Type

String literal in smalldatetime format

'yyyy-MM-dd hh:mm' or 'yyyy-MM-dd hh:mm:ss'

Example: '2007-05-08 12:00' or '2007-05-08 12:00:15'

The source data must have values for year, month, date, hour and minute. Seconds are optional and, if present, must be set to the value 00. Any other value generates an error.

Seconds are optional. When loading into a smalldatetime column, dwloader will round up seconds and fractional seconds. For example, 1999-01-05 20:10:35.123 will load as 01-05 20:11.

String literal in date format

'yyyy-MM-dd'

Example: '2007-05-08'

Time values (hour, minutes, seconds, and fractions) are set to 0 when the value is inserted.

date Data Type

The following table defines the default format and rules for loading literal values into a column of type date. An empty string ('') is converted to the default value '1900-01-01'. Strings that contain only blanks (' ') generate an error.

Input Data Type | Input Data Examples | Conversion to date Data Type
String literal in date format | 'yyyy-MM-dd' Example: '2007-05-08' |

time Data Type

The following table defines the default format and rules for loading literal values into a column of type time. An empty string ('') is converted to the default value '00:00:00.0000'. Strings that contain only blanks (' ') generate an error.

Input Data Type | Input Data Examples | Conversion to time Data Type
String literal in time format | 'hh:mm:ss.fffffff' Example: '12:35:29.1234567' | If the data source has a smaller or equal precision (number of fractional digits) than the precision of the time data type, the data is padded to the right with zeros. For example, a literal value '12:35:29.123' is inserted as '12:35:29.1230000'.

datetimeoffset Data Type

The following table defines the default format and rules for loading literal values into a column of type datetimeoffset (n). The default format is 'yyyy-MM-dd hh:mm:ss.fffffff {+|-}hh:mm'. An empty string ('') is converted to the default value '1900-01-01 12:00:00.0000000 +00:00'. Strings that contain only blanks (' ') generate an error. The number of fractional digits depends on the column definition. For example, a column defined as datetimeoffset (2) will have two fractional digits.

Input Data Type | Input Data Examples | Conversion to datetimeoffset Data Type
String literal in datetime format | 'yyyy-MM-dd hh:mm:ss[.fff]' Example: '2007-05-08 12:35:29.123' | Missing fractional digits and offset values are set to 0 when the value is inserted. For example, the literal '2007-05-08 12:35:29.123' is inserted as '2007-05-08 12:35:29.1230000 +00:00'.
String literal in smalldatetime format | 'yyyy-MM-dd hh:mm' Example: '2007-05-08 12:35' | Seconds, remaining fractional digits and offset values are set to 0 when the value is inserted.
String literal in date format | 'yyyy-MM-dd' Example: '2007-05-08' | Time values (hour, minutes, seconds, and fractions) are set to 0 when the value is inserted. For example, the literal '2007-05-08' is inserted as '2007-05-08 00:00:00.0000000 +00:00'.
String literal in datetime2 format | 'yyyy-MM-dd hh:mm:ss.fffffff' Example: '2007-05-08 12:35:29.1234567' | The source data cannot exceed the specified number of fractional seconds in the datetimeoffset column. If the data source has a smaller or equal number of fractional seconds, the data is padded to the right with zeros. For example, if the data type is datetimeoffset (5), the literal value '2007-05-08 12:35:29.123 +12:15' is inserted as '12:35:29.12300 +12:15'.
String literal in datetimeoffset format | 'yyyy-MM-dd hh:mm:ss.fffffff {+|-} hh:mm' Example: '2007-05-08 12:35:29.1234567 +12:15' | The source data cannot exceed the specified number of fractional seconds in the datetimeoffset column. If the data source has a smaller or equal number of fractional seconds, the data is padded to the right with zeros. For example, if the data type is datetimeoffset (5), the literal value '2007-05-08 12:35:29.123 +12:15' is inserted as '12:35:29.12300 +12:15'.

datetime2 Data Type

The following table defines the default format and rules for loading literal values into a column of type datetime2 (n). The default format is 'yyyy-MM-dd hh:mm:ss.fffffff'. An empty string ('') is converted to the default value '1900-01-01 12:00:00'. Strings that contain only blanks (' ') generate an error. The number of fractional digits depends on the column definition. For example, a column defined as datetime2 (2) will have two fractional digits.

Input Data Type | Input Data Examples | Conversion to datetime2 Data Type
String literal in datetime format | 'yyyy-MM-dd hh:mm:ss[.fff]' Example: '2007-05-08 12:35:29.123' | Fractional seconds are optional and are set to 0 when the value is inserted.
String literal in smalldatetime format | 'yyyy-MM-dd hh:mm' Example: '2007-05-08 12' | Optional seconds and remaining fractional digits are set to 0 when the value is inserted.
String literal in date format | 'yyyy-MM-dd' Example: '2007-05-08' | Time values (hour, minutes, seconds, and fractions) are set to 0 when the value is inserted. For example, the literal '2007-05-08' is inserted as '2007-05-08 12:00:00.0000000'.
String literal in datetime2 format | 'yyyy-MM-dd hh:mm:ss:fffffff' Example: '2007-05-08 12:35:29.1234567' | If the data source contains data and time components that are less than or equal to the value specified in datetime2(n), the data is inserted; otherwise an error is generated.

[TOP]

Datetime Formats

Dwloader supports the following data formats for the input data that it is loading into SQL Server PDW. More details are listed after the table.

Format | datetime | smalldatetime | date | datetime2 | datetimeoffset
mdy | [M[M]]M-[d]d-[yy]yy HH:mm:ss[.fff] | [M[M]]M-[d]d-[yy]yy HH:mm[:00] | [M[M]]M-[d]d-[yy]yy | [M[M]]M-[d]d-[yy]yy HH:mm:ss[.fffffff] | [M[M]]M-[d]d-[yy]yy HH:mm:ss[.fffffff] zzz
 | [M[M]]M-[d]d-[yy]yy hh:mm:ss[.fff][tt] | [M[M]]M-[d]d-[yy]yy hh:mm[:00][tt] |  | [M[M]]M-[d]d-[yy]yy hh:mm:ss[.fffffff][tt] | [M[M]]M-[d]d-[yy]yy hh:mm:ss[.fffffff][tt] zzz
myd | [M[M]]M-[yy]yy-[d]d HH:mm:ss[.fff] | [M[M]]M-[yy]yy-[d]d HH:mm[:00] | [M[M]]M-[yy]yy-[d]d | [M[M]]M-[yy]yy-[d]d HH:mm:ss[.fffffff] | [M[M]]M-[yy]yy-[d]d HH:mm:ss[.fffffff] zzz
 | [M[M]]M-[yy]yy-[d]d hh:mm:ss[.fff][tt] | [M[M]]M-[yy]yy-[d]d hh:mm[:00][tt] |  | [M[M]]M-[yy]yy-[d]d hh:mm:ss[.fffffff][tt] | [M[M]]M-[yy]yy-[d]d hh:mm:ss[.fffffff][tt] zzz
ymd | [yy]yy-[M[M]]M-[d]d HH:mm:ss[.fff] | [yy]yy-[M[M]]M-[d]d HH:mm[:00] | [yy]yy-[M[M]]M-[d]d | [yy]yy-[M[M]]M-[d]d HH:mm:ss[.fffffff] | [yy]yy-[M[M]]M-[d]d HH:mm:ss[.fffffff] zzz
 | [yy]yy-[M[M]]M-[d]d hh:mm:ss[.fff][tt] | [yy]yy-[M[M]]M-[d]d hh:mm[:00][tt] |  | [yy]yy-[M[M]]M-[d]d hh:mm:ss[.fffffff][tt] | [yy]yy-[M[M]]M-[d]d hh:mm:ss[.fffffff][tt] zzz
ydm | [yy]yy-[d]d-[M[M]]M HH:mm:ss[.fff] | [yy]yy-[d]d-[M[M]]M HH:mm[:00] | [yy]yy-[d]d-[M[M]]M | [yy]yy-[d]d-[M[M]]M HH:mm:ss[.fffffff] | [yy]yy-[d]d-[M[M]]M HH:mm:ss[.fffffff] zzz
 | [yy]yy-[d]d-[M[M]]M hh:mm:ss[.fff][tt] | [yy]yy-[d]d-[M[M]]M hh:mm[:00][tt] |  | [yy]yy-[d]d-[M[M]]M hh:mm:ss[.fffffff][tt] | [yy]yy-[d]d-[M[M]]M hh:mm:ss[.fffffff][tt] zzz
dmy | [d]d-[M[M]]M-[yy]yy HH:mm:ss[.fff] | [d]d-[M[M]]M-[yy]yy HH:mm[:00] | [d]d-[M[M]]M-[yy]yy | [d]d-[M[M]]M-[yy]yy HH:mm:ss[.fffffff] | [d]d-[M[M]]M-[yy]yy HH:mm:ss[.fffffff] zzz
 | [d]d-[M[M]]M-[yy]yy hh:mm:ss[.fff][tt] | [d]d-[M[M]]M-[yy]yy hh:mm[:00][tt] |  | [d]d-[M[M]]M-[yy]yy hh:mm:ss[.fffffff][tt] | [d]d-[M[M]]M-[yy]yy hh:mm:ss[.fffffff][tt] zzz
dym | [d]d-[yy]yy-[M[M]]M HH:mm:ss[.fff] | [d]d-[yy]yy-[M[M]]M HH:mm[:00] | [d]d-[yy]yy-[M[M]]M | [d]d-[yy]yy-[M[M]]M HH:mm:ss[.fffffff] | [d]d-[yy]yy-[M[M]]M HH:mm:ss[.fffffff] zzz
 | [d]d-[yy]yy-[M[M]]M hh:mm:ss[.fff][tt] | [d]d-[yy]yy-[M[M]]M hh:mm[:00][tt] |  | [d]d-[yy]yy-[M[M]]M hh:mm:ss[.fffffff][tt] | [d]d-[yy]yy-[M[M]]M hh:mm:ss[.fffffff][tt] zzz

Details:

  • To separate month, day and year values, you can use ' – ', ' / ', or ' . '. For simplicity, the table uses only the ' – ' separator.

  • To specify the month as text use three or more characters. Months with 1 or 2 characters will be interpreted as a number.

  • To separate time values, use the ' : ' symbol.

  • Letters enclosed in square brackets are optional.

  • The letters 'tt' designate [AM|PM|am|pm]. AM is the default. When 'tt' is specified, the hour value (hh) must be in the range of 0 to 12.

  • The letters 'zzz' designate the time zone offset for the system's current time zone in the format {+|-}HH:ss].

[TOP]

Inserting Literals into Numeric Types

The following tables define the default format and conversion rules for loading a literal value into a SQL Server PDW column that uses a numeric type.

bit Data Type

The following table defines the default format and rules for loading literal values into a column of type bit. An empty string ('') or a string that contain only blanks (' ') is converted to 0.

Input Data Type | Input Data Examples | Conversion to bit Data Type
String literal in integer format | 'ffffffffff' Example: '1' or '321' | An integer value formatted as a string literal cannot contain a negative value. For example, the value '-123' generates an error. A value larger than 1 is converted to 1. For example, the value '123' is converted to 1.
String literal | 'TRUE' or 'FALSE' Example: 'true' | The value 'TRUE' is converted to 1; the value 'FALSE' is converted to 0.
Integer literal | fffffffn Example: 1 or 321 | A value larger than 1 or less than 0 is converted to 1. For example, the values 123 and -123 are converted to 1.
Decimal literal | fffnn.fffn Example: 1234.5678 | A value larger than 1 or less than 0 is converted to 1. For example, the values 123.45 and -123.45 are converted to 1.

decimal Data Type

The following table defines the rules for loading literal values into a column of type decimal (p,s). Data conversion rules are the same as for SQL Server. For more information, see Data Type Conversion (Database Engine) on MSDN.

Input Data Type | Input Data Examples
Integer literal | 321312313123
Decimal literal | 123344.34455

float and real Data Types

The following table defines rules for loading literal values into a column of type float or real. Data conversion rules are the same as for SQL Server. For more information, see Data Type Conversion (Database Engine) on MSDN.

Input Data Type | Input Data Examples
Integer literal | 321312313123
Decimal literal | 123344.34455
Floating point literal | 3.12323E+14

int, bigint, tinyint, smallint Data Types

The following table defines the rules for loading literal values into a column of type int, bigint, tinyint, or smallint. The data source cannot exceed the range allowed for the given data type. For example, the range for tinyint is 0 to 255 and the range for int is -2,147,483,648 to 2,147,483,647.

Input Data Type | Input Data Examples | Conversion to Integer Data Types
Integer literal | 321312313123 |
Decimal literal | 123344.34455 | The values to the right of the decimal point are truncated.

money and smallmoney Data Types

Money literal values are represented as a string of numbers with an optional decimal point and an optional currency symbol as a prefix. The data source cannot exceed the range allowed for the given data type. For example, the range for smallmoney is -214,748.3648 to 214,748.3647 and the range for money is -922,337,203,685,477.5808 to 922,337,203,685,477.5807. The following table defines the rules for loading literal values into a column of type money or smallmoney.

Input Data Type | Input Data Examples | Conversion To money or smallmoney Data Type
Integer literal | 321312 | Missing digits after the decimal point are set to 0 when the value is inserted. For example, the literal 12345 is inserted as 12345.0000
Decimal literal | 123344.34455 | If the number of digits after the decimal point exceed 4, the value is rounded up to the nearest value. For example, the value 123344.34455 is inserted as 123344.3446.
Money literal | $123456.7890 | The currency symbol is not inserted with the value. If the number of digits after the decimal point exceed 4, the value is rounded up to the nearest value.

[TOP]

Inserting Literals into String Types

The following tables define the default format and conversion rules for loading a literal value into a SQL Server PDW column that uses a string type.

char, varchar, nchar and nvarchar Data Types

The following table defines the default format and rules for loading literal values into a column of type char, varchar, nchar and nvarchar. The data source length cannot exceed the size specified for the data type. If the data source length is less than size of the char or nchar data type, the data is padded to the right with blank spaces to reach the data type size.

Input Data Type

Input Data Examples

Conversion to Character Data Types

String literal

Format: 'character string'

Example: 'abc'

Unicode string literal

Format: N'character string'

Example: N'abc'

Integer literal

Format: ffffffffffn

Example: 321312313123

Decimal literal

Format: ffffff.fffffff

Example: 12344.34455

Money literal

Format: $ffffff.fffnn

Example: $123456.99

The optional currency symbol is not inserted with the value. To insert the currency symbol, insert the value as a string literal. This will match the format of the loader, which treats every literal as a string literal.

Commas are not allowed.

If the number of digits after the decimal point exceed 2, the value is rounded up to the nearest value. For example, the value 123.946789 is inserted as 123.95.

Only the default style 0 (no commas and 2 digits after the decimal point) is allowed when using the CONVERT function to insert money literals.

[TOP]

General Remarks

dwloader performs the same implicit conversions that SMP SQL Server performs, but does not support all of the implicit conversions that SMP SQL Server supports.

See Also

Concepts

Grant Permissions to Load Data (SQL Server PDW)


Load Data With INSERT (SQL Server PDW)

[image]

Load Data With INSERT (SQL Server PDW)
See Also Send Feedback

You can use the SQL INSERT statement to load data into a SQL Server PDW distributed or replicated table. For more information about INSERT, see INSERT (SQL Server PDW). For replicated tables and all non-distribution columns in a distributed table, SQL Server PDW uses SQL Server to implicitly convert the data values specified in the statement to the data type of the destination column. For more information about SQL Server data conversion rules, see Data Type Conversion (Database Engine) on MSDN. However, for distribution columns, SQL Server PDW supports only a subset of implicit conversions that SQL Server supports. Therefore, when you use the INSERT statement to load data into a distribution column, the source data must be specified in one of the formats defined in the following tables.

Contents

  • Inserting Literals into Binary Types

  • Inserting Literals into Date and Time Types

  • Inserting Literals into Numeric Types

  • Inserting Literals into String Types

Inserting Literals into Binary Types

The following table defines the accepted literal types, format, and conversion rules for inserting a literal value into a distribution column of type binary (n) or varbinary(n).

Literal Type | Format | Conversion Rules
Binary literal | 0xhexidecimal_string Example: 0x12Ef | Binary literals must be prefixed with 0x. The data source length cannot exceed the number of bytes specified for the data type. If the data source length is less than size of the binary data type, the data is padded to the right with zeros to reach the data type size.

[TOP]

Inserting Literals into Date and Time Types

Date and time literals are represented by using character values in specific formats, enclosed in single quotation marks. The following tables define the allowed literal types, format, and conversion rules for inserting a date or time literal into a SQL Server PDW distribution column of type datetime, smalldatetime, date, time, datetimeoffset, or datetime2.

datetime Data Type

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type datetime. Any empty string ('') is converted to the default value '1900-01-01 12:00:00.000'. Strings that contain only blanks (' ') generate an error.

Literal Type | Format | Conversion Rules
String literal in datetime format | 'YYYY-MM-DD hh:mm:ss[.nnn]' Example: '2007-05-08 12:35:29.123' | Missing fractional digits are set to 0 when the value is inserted. For example, the literal '2007-05-08 12:35' is inserted as '2007-05-08 12:35:00.000'.
String literal in smalldatetime format | 'YYYY-MM-DD hh:mm' Example: '2007-05-08 12:35' | Seconds and remaining fractional digits are set to 0 when the value is inserted.
String literal in date format | 'YYYY-MM-DD' Example: '2007-05-08' | Time values (hour, minutes, seconds, and fractions) are set to 12:00:00.000 when the value is inserted.
String literal in datetime2 format | 'YYYY-MM-DD hh:mm:ss.nnnnnnn' Example: '2007-05-08 12:35:29.1234567' | The source data cannot exceed three fractional digits. For example, the literal '2007-05-08 12:35:29.123' will be inserted, but the value '2007-05-08 12:35:29.1234567' generates an error.

smalldatetime Data Type

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type smalldatetime. Any empty string ('') is converted to the default value '1900-01-01 12:00'. Strings that contain only blanks (' ') generate an error.

Literal Type | Format | Conversion Rules
String literal in smalldatetime format | 'YYYY-MM-DD hh:mm' or 'YYYY-MM-DD hh:mm:00' Example: '2007-05-08 12:00' or '2007-05-08 12:00:00' | The source data must have values for year, month, date, hour and minute. Seconds are optional and, if present, must be set to the value 00. Any other value generates an error.
String literal in date format | 'YYYY-MM-DD' Example: '2007-05-08' | Time values (hour, minutes, seconds, and fractions) are set to 0 when the value is inserted.

date Data Type

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type date. Any empty string ('') is converted to the default value '1900-01-01'. Strings that contain only blanks (' ') generate an error.

Literal Type | Format | Conversion Rules
String literal in date format | 'YYYY-MM-DD' Example: '2007-05-08' | This is the only accepted format.

time Data Type

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type time. Any empty string ('') is converted to the default value '00:00:00.0000'. Strings that contain only blanks (' ') generate an error.

Literal Type | Format | Conversion Rules
String literal in time format | 'hh:mm:ss.nnnnnnn' Example: '12:35:29.1234567' | If the data source has a smaller or equal precision (number of fractional digits) than the precision of the time data type, the data is padded to the right with zeros. For example, a literal value '12:35:29.123' is inserted as '12:35:29.1230000'. A value that has a larger precision than the target data type is rejected.

datetimeoffset Data Type

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type datetimeoffset (n). The default format is 'YYYY-MM-DD hh:mm:ss.nnnnnnn {+|-}hh:mm'. An empty string ('') is converted to the default value '1900-01-01 12:00:00.0000000 +00:00'. Strings that contain only blanks (' ') generate an error. The number of fractional digits depends on the column definition. For example, a column defined as datetimeoffset (2) will have two fractional digits.

Literal Type | Format | Conversion Rules
String literal in datetime format | 'YYYY-MM-DD hh:mm:ss[.nnn]' Example: '2007-05-08 12:35:29.123' | Missing fractional digits and offset values are set to 0 when the value is inserted. For example, the literal '2007-05-08 12:35:29.123' is inserted as '2007-05-08 12:35:29.1230000 +00:00'.
String literal in smalldatetime format | 'YYYY-MM-DD hh:mm' Example: '2007-05-08 12:35' | Seconds, remaining fractional digits and offset values are set to 0 when the value is inserted.
String literal in date format | 'YYYY-MM-DD' Example: '2007-05-08' | Time values (hour, minutes, seconds, and fractions) are set to 0 when the value is inserted. For example, the literal '2007-05-08' is inserted as '2007-05-08 00:00:00.0000000 +00:00'.
String literal in datetime2 format | 'YYYY-MM-DD hh:mm:ss.nnnnnnn' Example: '2007-05-08 12:35:29.1234567' | The source data cannot exceed the specified number of fractional seconds in the datetimeoffset column. If the data source has a smaller or equal number of fractional seconds, the data is padded to the right with zeros. For example, if the data type is datetimeoffset (5), the literal value '2007-05-08 12:35:29.123 +12:15' is inserted as '12:35:29.12300 +12:15'.
String literal in datetimeoffset format | 'YYYY-MM-DD hh:mm:ss.nnnnnnn {+|-} hh:mm' Example: '2007-05-08 12:35:29.1234567 +12:15' | The source data cannot exceed the specified number of fractional seconds in the datetimeoffset column. If the data source has a smaller or equal number of fractional seconds, the data is padded to the right with zeros. For example, if the data type is datetimeoffset (5), the literal value '2007-05-08 12:35:29.123 +12:15' is inserted as '12:35:29.12300 +12:15'.

datetime2 Data Type

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type datetime2 (n). The default format is 'YYYY-MM-DD hh:mm:ss.nnnnnnn'. An empty string ('') is converted to the default value '1900-01-01 12:00:00'. Strings that contain only blanks (' ') generate an error. The number of fractional digits depends on the column definition. For example, a column defined as datetime2 (2) will have two fractional digits.

Literal Type | Format | Conversion Rules
String literal in datetime format | 'YYYY-MM-DD hh:mm:ss[.nnn]' Example: '2007-05-08 12:35:29.123' | Fractional seconds are optional and are set to 0 when the value is inserted. A value that has more fractional digits than the target data type is rejected.
String literal in smalldatetime format | 'YYYY-MM-DD hh:mm' Example: '2007-05-08 12' | Optional seconds and remaining fractional digits are set to 0 when the value is inserted.
String literal in date format | 'YYYY-MM-DD' Example: '2007-05-08' | Time values (hour, minutes, seconds, and fractions) are set to 0 when the value is inserted. For example, the literal '2007-05-08' is inserted as '2007-05-08 12:00:00.0000000'.
String literal in datetime2 format | 'YYYY-MM-DD hh:mm:ss:nnnnnnn' Example: '2007-05-08 12:35:29.1234567' | If the data source contains data and time components that are less than or equal to the value specified in datetime2(n), the data is inserted; otherwise an error is generated.

[TOP]

Inserting Literals into Numeric Types

The following tables define the accepted formats and conversion rules for inserting a literal value into a SQL Server PDW distribution column that uses a numeric type.

bit Data Type

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type bit. An empty string ('') or a string that contain only blanks (' ') is converted to 0.

Literal Type | Format | Conversion Rules
String literal in integer format | 'nnnnnnnnnn' Example: '1' or '321' | An integer value formatted as a string literal cannot contain a negative value. For example, the value '-123' generates an error. A value larger than 1 is converted to 1. For example, the value '123' is converted to 1.
String literal | 'TRUE' or 'FALSE' Example: 'true' | The value 'TRUE' is converted to 1; the value 'FALSE' is converted to 0.
Integer literal | nnnnnnnn Example: 1 or 321 | A value larger than 1 or less than 0 is converted to 1. For example, the values 123 and -123 are converted to 1.
Decimal literal | nnnnn.nnnn Example: 1234.5678 | A value larger than 1 or less than 0 is converted to 1. For example, the values 123.45 and -123.45 are converted to 1.

decimal Data Type

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type decimal (p,s). The data conversion rules are the same as for SQL Server. For more information, see Data Type Conversion (Database Engine) on MSDN.

Literal Type | Format
String literal in integer format | 'nnnnnnnnnnnn' Example: '321312313123'
String literal in decimal format | 'nnnnnn.nnnnn' Example: '123344.34455'
Integer literal | nnnnnnnnnnnn Example: 321312313123
Decimal literal | nnnnnn.nnnnn Example: '123344.34455'

float and real Data Types

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type float or real. Data conversion rules are the same as for SQL Server. For more information, see Data Type Conversion (Database Engine) on MSDN.

Literal Type | Format
String literal in integer format | 'nnnnnnnnnnnn' Example: '321312313123'
String literal in decimal format | 'nnnnnn.nnnnn' Example: '123344.34455'
String literal in floating point format | 'n.nnnnnE+nn' Example: '3.12323E+14'
Integer literal | nnnnnnnnnnnn Example: 321312313123
Decimal literal | nnnnnn.nnnnn Example: 123344.34455
Floating point literal | n.nnnnnE+nn Example: 3.12323E+14

int, bigint, tinyint, smallint Data Types

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type int, bigint, tinyint, or smallint. The data source cannot exceed the range allowed for the given data type. For example, the range for tinyint is 0 to 255 and the range for int is -2,147,483,648 to 2,147,483,647.

Literal Type | Format | Conversion Rules
String literal in integer format | 'nnnnnnnnnnnnnn' Example: '321312313123' |
Integer literal | nnnnnnnnnnnnnn Example: 321312313123 |
Decimal literal | nnnnnn.nnnnn Example: 123344.34455 | The values to the right of the decimal point are truncated.

money and smallmoney Data Types

Money literal values are represented as numbers with an optional decimal point and currency symbol as a prefix. The data source cannot exceed the range allowed for the given data type. For example, the range for smallmoney is -214,748.3648 to 214,748.3647 and the range for money is -922,337,203,685,477.5808 to 922,337,203,685,477.5807. The following table defines the accepted formats and rules for inserting literal values into a distribution column of type money or smallmoney.

Literal Type | Format | Conversion Rules
String literal in integer format | 'nnnnnnnn' Example: '123433' | Missing digits after the decimal point are set to 0 when the value is inserted. For example, the literal '12345' is inserted as 12345.0000.
String literal in decimal format | 'nnnnnn.nnnnn' Example: '123344.34455' | If the number of digits after the decimal point exceed 4, the value is rounded up to the nearest value. For example, the value '123344.34455' is inserted as 123344.3446.
String literal in money format | '$nnnnnn.nnnn' Example: '$123456.7890' | The optional currency symbol is not inserted with the value. If the number of digits after the decimal point exceed 4, the value is rounded up to the nearest value.
Integer literal | nnnnnnnn Example: 123433 | Missing digits after the decimal point are set to 0 when the value is inserted. For example, the literal 12345 is inserted as 12345.0000.
Decimal literal | nnnnnn.nnnnn Example: 123344.34455 | If the number of digits after the decimal point exceed 4, the value is rounded up to the nearest value. For example, the value 123344.34455 is inserted as 123344.3446.
Money literal | $nnnnnn.nnnn Example: $123456.7890 | The optional currency symbol is not inserted with the value. If the number of digits after the decimal point exceed 4, the value is rounded up to the nearest value.

[TOP]

Inserting Literals into String Types

The following tables define the accepted formats and conversion rules for inserting a literal value into a SQL Server PDW column that uses a string type.

char, varchar, nchar and nvarchar Data Types

The following table defines the accepted formats and rules for inserting literal values into a distribution column of type char, varchar, nchar and nvarchar. The data source length cannot exceed the size specified for the data type. If the data source length is less than size of the char or nchar data type, the data is padded to the right with blank spaces to reach the data type size.

Literal Type

Format

Conversion Rules

String literal

Format: 'character string'

Example: 'abc'

Unicode string literal

Format: N'character string'

Example: N'abc'

Integer literal

Format: nnnnnnnnnnn

Example: 321312313123

Decimal literal

Format: nnnnnn.nnnnnnn

Example: 12344.34455

Money literal

Format: $nnnnnn.nnnnn

Example: $123456.99

The currency symbol is not inserted with the value. To insert the currency symbol, insert the value as a string literal. This will match the format of the dwloader tool, which treats every literal as a string literal.

Commas are not allowed.

If the number of digits after the decimal point exceed 2, the value is rounded up to the nearest value. For example, the value 123.946789 is inserted as 123.95.

Only the default style 0 (no commas and 2 digits after the decimal point) is allowed when using the CONVERT function to insert money literals.

[TOP]

See Also

Reference

INSERT (SQL Server PDW)

Concepts

Grant Permissions to Load Data (SQL Server PDW)
Common Metadata Query Examples (SQL Server PDW)
Distributed and Replicated Tables (SQL Server PDW)


Load Data with Prepared Statements (SQL Server PDW)

[image]

Load Data with Prepared Statements (SQL Server PDW)
See Also Send Feedback

You can use SQL Server PDW prepared statements to load data into distributed and replicated tables. When the input data does not match the target data type, an implicit conversion is performed. The implicit conversions supported by SQL Server PDW prepared statements are a subset of conversions supported by SQL Server. That is, only a subset of conversions are supported, but the supported conversions match SQL Server implicit conversions. Regardless of whether the target table to be loaded is defined as a distributed or replicated table, implicit conversions are applied (if needed) to all columns that exist in target table. For more information about prepared statements, see Running Prepared Statements (SQL Server PDW).

See Also


dwloader Command-Line Loader (SQL Server PDW)

[image]

dwloader Command-Line Loader (SQL Server PDW)
See Also Send Feedback

dwloader is a SQL Server PDW command-line tool that loads table rows in bulk into an existing table. When loading rows, you can add all rows to the end of the table (append mode or fastappend mode), append new rows and update existing rows (upsert mode), or delete all existing rows before loading and then insert all rows into an empty table (reload mode).

Process for loading data

  1. Prepare the source data.

    Use your own ETL process to create the source data you wish to load. The source data must be formatted to match the schema of the destination table. Store the source data into one or more text files and copy the text files into the same directory on your Loading server. For information about the Loading server, see Acquire and Configure a Loading Server (SQL Server PDW)

  2. Prepare the loading options.

    Decide which loading options you will use. Store the load options in a configuration file. Copy the configuration file to a local location on your Loading server. The dwloader configuration options are described in this topic.

  3. Prepare the load failure options.

    Decide how you want dwloader to handle rows that fail to load. To perform the load, dwloader first loads the data into a staging table and then transfers the data to the destination table. As the Loader loads data into the staging table, it tracks the number of rows that fail to load. For example, rows that are not formatted correctly will fail to load. Failed rows are copied to a rejection file. By default, the load aborts after the first rejection unless you specify a different rejection threshold.

  4. Install dwloader.

    Install dwloader onto the Loading server if it is not already installed. See Install dwloader Command-Line Loader (SQL Server PDW).

  5. Run dwloader.

    Log in to the Loading server and run the executable dwloader.exe with the appropriate command-line options.

  6. Verify the results.

    You can check the failed rows file (specified with -R) to see if any rows failed to load. If this file is empty, all rows loaded successfully. dwloader is transactional, so if any step fails (other than rejected rows), all steps will roll back to their initial state.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

dwloader.exe { -h }

dwloader.exe
    {
        { -U login_name  -P password  }
        | -W
    }
    [ -f parameter_file ]
    [ -S target_appliance ]
    { -T target_database_name . [ schema ] . table_name }
    { -i source_data_location } [ <source_data_options> ]
    { -R load_failure_file_name } [ <load_failure_options> ]
    [ <loading_options> ]
}

<source_data_options> ::=
{
    [ -fh number_header_rows ]
    [ < variable_length_column_options > | < fixed_width_column_options > ]
    [ -D { mdy | myd | ymd | ydm | dmy | dym | custom_date_format } ]
    [ -dt datetime_format_file ]
}

<variable_length_column_options> ::=
{
    [ -e character_encoding ]
    -r row_delimiter
    [ -s string_delimiter ]
    -t field_delimiter
}

<fixed_width_column_options> ::=
{
    -w fixed_width_config_file
    [ -e character_encoding ]
    -r row_delimiter
}

<load_failure_options> ::=
{
    [ -rt { value | percentage } ]
    [ -rv reject_value ]
    [ -rs reject_sample_value ]
}

<loading_options> ::=
{
    [ -d staging_database_name ]
    [ -M { append | fastappend | upsert -K merge_column [ ,...n ] | reload } ]
    [ -b batchsize ]
    [ -c ]
    [ -E ]
    [ -m ]
    [ -N ]
    [ -se ]
}

Arguments

-h

Displays simple Help information about using the Loader. Help only displays if no other command-line parameters are specified.

-U login_name

A valid SQL Server Authentication login with appropriate permissions to perform the load.

-P password

The password for a SQL Server Authentication login_name.

-W

Use Windows Authentication. (No login_name or password required.) For information about configuring Windows Authentication, see Security - Configure Domain Trusts (SQL Server PDW).

-f parameter_file_name

Use a parameter file, parameter_file_name, in place of command-line parameters. parameter_file_name can contain any command-line parameter except user_name and password. If a parameter is specified on the command line and in the parameter file, the command line overrides the file parameter.

The parameter file contains one parameter, without the - prefix, per line.

Examples:

rt=percentage

rv=25

-S target_appliance

Specifies the SQL Server PDW appliance that will receive the loaded data.

For Infiniband connections, target_appliance is specified as <appliance-name>-SQLCTL01. To configure this named connection, see Configure InfiniBand Network Adapters (SQL Server PDW).

For Ethernet connections, target_appliance is the IP address for the Control node cluster.

If omitted, dwloader defaults to the value that was specified when dwloader was installed. For more information about this install option, see Install dwloader Command-Line Loader (SQL Server PDW).

-T target_database_name.[schema].table_name

The three-part name for the destination table.

-I source_data_location

The location of one or more source files to load. Each source file must be a text file or a text file that is compressed with gzip. Only one source file can be compressed into each gzip file.

To format a source file:

  • The source file must be formatted in accordance with the load options.

  • Each line in a source file contains the data for one table row. The source data must match the schema of the destination table. Column order and data types must also match. Each field in the row represents a column in the destination table.

  • By default, the fields are variable length and separated by a delimiter. To specify the delimiter type, use the <variable_length_column_options> command-line options. To specify fixed length fields, use the <fixed_width_column_options> command-line options.

To specify the source data location:

  • The source data location can be a network path or a local path to a directory on the loading server.

  • To specify all the files in a directory, enter the directory path followed by the * wildcard character. The loader does not load files from any subdirectories that are in the source data location.. The loader errors when a directory exists in a gzip file.

  • To specify some of the files in a directory, use a combination of characters and the * wildcard.

To load multiple files with one command:

  • All files must exist in the same directory.

  • The files must be all text files, all gzip files, or a combination of both text and gzip files.

  • None of the files can contain header information.

  • All files must use the same character encoding type. See the –e option.

  • All files must be loaded into the same table.

  • All files will be concatenated and loaded as though they are one file, and the rejected rows will go to a single reject file.

Examples:

  • -i \\loadserver\loads\daily\*.gz

  • -i \\loadserver\loads\daily\*.txt

  • -i \\loadserver\loads\daily\monday.*

  • -i \\loadserver\loads\daily\monday.txt

  • -i \\loadserver\loads\daily\*

-R load_failure_file_name

If there are load failures, dwloader stores the row that failed to load and the failure description the failure information in a file named load_failure_file_name. If this file already exists, dwloader will overwrite the existing file. load_failure_file_name is created when the first failure occurs. If all rows load successfully, load_failure_file_name is not created.

-fh number_header_rows

The number of lines (rows) to ignore at the beginning of source_data_file_name. The default is 0.

<variable_length_column_options>

The options for a source_data_file_name that has character-delimited variable-length columns. By default, source_data_file_name contains ASCII characters in variable-length columns.

For ASCII files, NULLs are represented by placing delimiters consecutively. For example, in a pipe-delimited file ("|"), a NULL is indicated by "||". In a comma-delimited file, a NULL is indicated by ",,". Additionally, the -E (--emptyStringAsNull) option must be specified. For more information on -E, see below.

-e character_encoding

Specifies a character-encoding type for the data to be loaded from the data file. Options are ASCII (default), UTF8, UTF16, or UTF16BE, where UTF16 is little endian and UTF16BE is big endian. These options are case insensitive.

-t field_delimiter

The delimiter for each field (column) in the row. The field delimiter is one or more of these ASCII escape characters or ASCII hex values..

Name

Escape Character

Hex Character

Tab

\t

0x09

Carriage return (CR)

\r

0x0d

Line feed (LF)

\n

0x0a

CRLF

\r\n

0x0d0x0a

Comma

','

0x2c

Double quote

\"

0x22

Single quote

\'

0x27

To specify the pipe character on the command line, enclose it with double quotes, "|". This will avoid misinterpretation by the command-line parser. Other characters are enclosed with single quotes.

Examples:

-t "|"

-t ' '

-t 0x0a

-t \t

-t '~|~'

-r row_delimiter

The delimiter for each row of the source data file. The row delimiter is one or more ASCII values.

To specify a carriage return (CR), linefeed (LF), or tab character as a delimiter, you can use the escape characters (\r, \n, \t) or their hex values (0x, 0d, 09). To specify any other special characters as delimiters, use their hex value.

Examples of CR+LF:

-r \r\n

-r 0x0d0x0a

Examples of CR:

-r \r

-r 0x0d

Examples of LF:

-r \n

-r 0x0a

An LF is required for Unix. A CR is required for Windows.

-s string_delimiter

The delimiter for string data type field of a text-delimited input file. The string delimiter is one character in length. It can be specified as a character (e.g., -s * ) or as a hex value (e.g., -s 0x22 for a double quote).

Examples:

-s *

-s 0x22

< fixed_width_column_options>

The options for a source data file that has fixed-length columns. By default, source_data_file_name contains ASCII characters in variable-length columns.

Fixed width columns are not supported when –e is UTF8.

-w fixed_width_config_file

Path and name of the configuration file that specifies the number of characters in each column. Every field must be specified.

This file must reside on the Loading server. The path can be a UNC, relative, or absolute path. Each line in fixed_width_config_file contains the name of one column and the number of characters for that column. There is one line per column, as follows, and the order in the file must match the order in the destination table:

column_name=num_chars

column_name=num_chars

Example fixed width config file:

SalesCode=3

SalesID=10

Example lines in source_data_file_name:

230Shirts0056

320Towels1356

In the previous example, the first loaded row will have SalesCode='230' and SalesID='Shirts0056'. The second loaded row will have SalesCode='320' and SaleID='Towels1356'.

For information on how to handle leading and trailing spaces or data type conversion in fixed width mode, see Data Type Conversion Rules for dwloader (SQL Server PDW).

-e character_encoding

Specifies a character-encoding type for the data to be loaded from the data file. Options are ASCII (default), UTF8, UTF16, or UTF16BE, where UTF16 is little endian and UTF16BE is big endian. These options are case insensitive.

Fixed width columns are not supported when –e is UTF8.

-r row_delimiter

The delimiter for each row of the source data file. The row delimiter is one or more ASCII values.

To specify a carriage return (CR), linefeed (LF), or tab character as a delimiter, you can use the escape characters (\r, \n, \t) or their hex values (0x, 0d, 09). To specify any other special characters as delimiters, use their hex value.

Examples of CR+LF:

-r \r\n

-r 0x0d0x0a

Examples of CR:

-r \r

-r 0x0d

Examples of LF:

-r \n

-r 0x0a

An LF is required for Unix. A CR is required for Windows.

-D { ymd | ydm | mdy | myd | dmy | dym | custom_date_format }

Specifies the order of month (m), day (d), and year (y) for all datetime fields in the input file. The default order is ymd. To specify multiple order formats for the same source file, use the –dt option.

ymd | dmy

ydm and dmy allow the same input formats. Both allow the year to be at the beginning or the end of the date. For example, for both ydm and dmy date formats, you could have 2013-02-03 or 02-03-2013 in the input file.

ydm

You can only load input formatted as ydm into columns of data type datetime and smalldatetime. You cannot load ydm values into a column of the datetime2, date, or datetimeoffset data type.

mdy

mdy allows <month><space><day><comma><year>.

Examples of mdy input data for January 1, 1975:

  • January 1, 1975

  • Jan 01, 75

  • Jan/1/75

  • 01011975

myd

Input file examples for March 04,2010: 03-2010-04, 3/2010/4

dym

Input file examples for March 04, 2010: 04-2010-03, 4/2010/3

custom_date_format

custom_date_format is a custom date format (e.g., MM/dd/yyyy ) and included for backward compatibility only. dwloader does not enfoce the custom date format. Instead, when you specify a custom date format, dwloader will convert it to the corresponding setting of ymd, ydm, mdy, myd, dym, or dmy.

For example, if you specify –D MM/dd/yyyy, dwloader expects all date input to be ordered with month first, then day, and then year (mdy). It does not enforce 2 character months, 2 digit days, and 4 digit years as specified by the custom date format. Here are some examples of ways dates can be formatted in the input file when the date format is –D MM/dd/yyyy: 01/02/2013, Jan.02.2013, 1/2/2013

For more comprehensive formatting information, see Data Type Conversion Rules for dwloader (SQL Server PDW).

-dt datetime_format_file

Each datetime format is specified in a file named datetime_format_file. Unlike the command-line parameters, file parameters that include spaces must not be enclosed in double quotes. You cannot alter the datetime format as you load data. The source data file and its corresponding column in the destination table must have the same format.

Each line contains the name of a column in the destination table and its datetime format.

Examples:

LastReceiptDate=ymd

ModifiedDate=dym

-d staging_database_name

The database name that will contain the staging table. The default is the database specified with the -T option, which is the database for the destination table. For more information about using a staging database, see Create the Staging Database (SQL Server PDW).

-M load_mode_option

Specifies whether to append, upsert, or reload data. The default mode is append.

append

The Loader inserts rows at the end of existing rows in the destination table.

fastappend

The Loader inserts rows directly, without using a temporary table, to the end of existing rows in the destination table. fastappend requires the multi-transaction (–m) option. A staging database cannot be specified when using fastappend. There is no rollback with fastappend, which means that recovery from a failed or aborted load must be handled by your own load process.

upsert -K merge_column [ ,...n ]

The Loader uses the SQL Server Merge statement to update existing rows and insert new rows.

The -K option specifies the column or columns to base the merge on. These columns form a merge key, which should represent a unique row. If the merge key exists in the destination table, the row is updated. If the merge key doesn't exist in the destination table, the row is appended.

For hash-distributed tables, the merge key must be or include the distribution column.

For replicated tables, the merge key is the combination of one or more columns. These columns are specified according to the needs of the application.

Multiple columns must be comma-separated with no spaces, or comma-separated with spaces and enclosed in single quotes.

If two rows in the source table have matching merge key values, their respective rows must be identical.

reload

The Loader truncates the destination table before it inserts the source data.

-b batchsize

Recommended only for use by Microsoft Support, batchsize is the SQL Server batch size for the bulk copy that DMS performs into SQL Server instances on the Compute nodes. When batchsize is specified, SQL Server PDW will override the batch load size that is calculated dynamically for each load.

Beginning with SQL Server 2012 PDW, the Control node dynamically computes a batch size for each load by default. This automatic calculation is based on several parameters such as memory size, target table type, target table schema, load type, file size, and the user’s resource class.

For example, if the load mode is FASTAPPEND and the table has a clustered columnstore index, SQL Server PDW will by default attempt to use a batch size of 1,048,576 so that rowgroups will become CLOSED and load directly into the columnstore without going through the delta store. If memory does not allow the batch size of 1,048,576, dwloader will choose a smaller batchsize.

If the load type is FASTAPPEND, the batchsize applies to loading data into the table, otherwise batchsize applies to loading data into the staging table.

<reject_options>

Specifies options for determining the number of load failures that the Loader will allow. If the load failures exceed the threshold, the loader will halt and not commit any rows.

-rt { value | percentage }

Specifies whether the -reject_value in the -rv reject_value option is a literal number of rows (value) or a rate of failure (percentage). The default is value.

The percentage option is a real-time calculation that occurs at intervals according to the -rs option.

For example, if the Loader attempts to load 100 rows and 25 fail and 75 succeed, then the failure rate is 25%.

-rv reject_value

Specifies the number or percentage of row rejections to allow before halting the load. The -rt option determines if reject_value refers to the number of rows or the percentage of rows.

The default reject_value is 0.

When used with -rt value, the loader stops the load when the rejected row count exceeds reject_value.

When use with -rt percentage, the loader computes the percentage at intervals (-rs option). Therefore, the percentage of failed rows can exceed reject_value.

-rs reject_sample_size

Used with the -rt percentage option to specify the incremental percentage checks. For example, if reject_sample_size is 1000, the Loader will calculate the percentage of failed rows after it has attempted to load 1000 rows. It recalculates the percentage of failed rows after it attempts to load each additional 1000 rows.

-c

Removes white space characters from the left and right side of char, nchar, varchar, and nvarchar fields. Converts each field that contains only white space characters to the empty string.

Examples:

' ' gets truncated to ''

' abc ' gets truncated to 'abc'

When –c is used with -E, the –E operation occurs first. Fields that contain only white space characters are converted to the empty string, and not to NULL.

-E

Convert empty strings to NULL. The default is to not perform these conversions.

-m

Use multi-transaction mode for the second phase of loading; when loading data from the staging table into a distributed table.

With –m, SQL Server PDW performs and commits loads in parallel. This performs much faster than the default loading mode, but is not transaction-safe.

Without –m, SQL Server PDW performs and commits loads serially across the distributions within each Compute node, and concurrently across the Compute nodes. This method is slower than multi-transaction mode, but is transaction-safe.

-m is optional for append, reload, and upsert.

-m is required for fastappend.

-m cannot be used with replicated tables.

-m applies only to the second loading phase. It does not apply to the first loading phase; loading data into the staging table.

There is no rollback with multi-transaction mode, which means that recovery from a failed or aborted load must be handled by your own load process.

We recommend using –m only when loading into an empty table, so that you can recover without data loss. To recover from a load failure: drop the destination table, resolve the load issue, re-create the destination table, and run the load again.

-N

Verify the target appliance has a valid SQL Server PDW certificate from a trusted authority. Use this to help ensure your data is not being hijacked by an attacker and sent to an unauthorized location. The certificate must already be installed on the appliance. The only supported way to install the certificate is for the appliance administrator to install it by using the Configuration Manager tool. Ask your appliance administrator if you are not sure whether the appliance has a trusted certificate installed.

-se

Skip loading empty files. This also skips uncompressing empty gzip files.

Return Code Values

0 (success) or other integer value (failure)

In a command window or batch file, use errorlevel to display the return code. For example:

Copy image Copy Code
dwloader
echo ReturnCode=%errorlevel%
if not %errorlevel%==0 echo Fail
if %errorlevel%==0 echo Success

When using PowerShell, use $LastExitCode.

Permissions

Requires LOAD permission and applicable permissions (INSERT, UPDATE, DELETE) on the destination table. Requires CREATE permission (for creating a temporary table) on the staging database. If a staging database is not used, then CREATE permission is required on the destination database. For more information, see Grant Permissions to Load Data (SQL Server PDW).

General Remarks

For information on data type conversions when loading with dwloader, see Data Type Conversion Rules for dwloader (SQL Server PDW).

If a parameter includes one or more spaces, enclose the parameter with double quotes.

You should run the Loader from its installed location. The dwloader executable is pre-installed with the appliance and is located in the C:\Program Files\Microsoft SQL Server Data Warehouse\DWLoader directory.

You can override a parameter that is specified in the parameter file (-f option) by specifying it as a command-line parameter.

You can run multiple instances of the Loader simultaneously. The maximum number of Loader instances is pre-configured and cannot be changed. For the maximum number of loads per appliance, see Minimum and Maximum Values (SQL Server PDW)

Loaded data might require more or less space on the appliance than in the source location. You can perform test imports with subsets of data to estimate disk consumption.

Although dwloader is a transaction process and will roll back gracefully on failure, it cannot be rolled back once the bulk load has been completed successfully. To cancel an active dwloader process, type CTRL+C.

Limitations and Restrictions

The total size of all loads occurring concurrently must be smaller than LOG_SIZE for the database, and we recommend the total size of all concurrent loads is less than 50% of the LOG_SIZE. To achieve this size limitation, you can split large loads into multiple batches. For more information on LOG_SIZE, see CREATE DATABASE (SQL Server PDW)

When loading multiple files with one load command, all rejected rows are written to the same reject file. The reject file does not show which input file contains each rejected row.

Locking Behavior

dwloader locking behavior varies depending on the load_mode_option.

  • append – Append is the recommended and the most common option. Append loads data into a staging table. The locking is described in detail below.

  • fast append – Fast-append loads directly into the final table taking an ExclusiveUpdate table lock, and is the only mode that does not use a staging table.

  • reload – Reload loads data into a staging table and requires an exclusive lock on both the staging table and the final table. Reload is not recommended for concurrent operations.

  • upsert – Upsert loads data into a staging table, and then performs a merge operation from the staging table to the final table. Upsert does not require exclusive locks on the final table. Performance may vary when using upsert. Test the behavior in your environment.

Locking Behavior

A append mode locking

Append can be run in multi-transactional mode (using the –m argument) but it is not transaction safe. Therefore append should be used as a transactional operation (without using the –m argument). Unfortunately, during the final INSERT-SELECT operation, transactional mode is currently about six times slower than the multi-transactional mode.

The append mode loads data in two phases. Phase one loads data from the source file into a staging table concurrently (fragmentation can occur). Phase two loads data from the staging table to the final table. The second phase performs an INSERT INTO…SELECT WITH (TABLOCK) operation. The following table shows the locking behavior on the final table, and logging behavior when using append mode:

Table Type | Multi-transaction Mode (-m) | Table is Empty | Concurrency Supported | Logging
Heap | Yes | Yes | Yes | Minimal
Heap | Yes | No | Yes | Minimal
Heap | No | Yes | No | Minimal
Heap | No | No | No | Minimal
Cl | Yes | Yes | No | Minimal
Cl | Yes | No | Yes | Full
Cl | No | Yes | No | Minimal
Cl | No | No | Yes | Full

The above table shows dwloader using the append mode loading into a heap or a clustered index (CI) table, with or without the multi-transactional flag, and loading into an empty table or a non-empty table. The locking and logging behavior of each such combination of load is displayed in the table. For instance, loading (2nd) phase with the append mode into a clustered index without multi-transactional mode and into an empty table will have PDW create an exclusive lock on the table and logging is minimal. This means that a customer will not be able to load (2nd) phase and query concurrently into an empty table. However, when loading with the same configuration into a non-empty table, PDW will not issue an exclusive lock on the table and concurrency is possible. Unfortunately, full logging occurs, slowing the process.

Examples

A. Simple dwloader example

The following example shows initiation of the Loader with only the required options selected. Other options are taken from the global configuration file, loadparamfile.txt.

Example using SQL Server Authentication.

Copy image Copy Code
--Load over Ethernet
dwloader.exe -S 10.192.63.148 -U mylogin -P 123jkl -f /configfiles/loadparamfile.txt

--Load over InfiniBand to appliance named MyPDW
dwloader.exe -S MyPDW-SQLCTL01 -U mylogin -P 123jkl -f /configfiles/loadparamfile.txt

The same example using Windows Authentication.

Copy image Copy Code
--Load over Ethernet
dwloader.exe -S 10.192.63.148 -W -f /configfiles/loadparamfile.txt

--Load over InfiniBand to appliance named MyPDW
dwloader.exe -S MyPDW-SQLCTL01 -W -f /configfiles/loadparamfile.txt

Example using arguments for a source file and error file.

Copy image Copy Code
--Load over Ethernet
dwloader.exe -U mylogin -P 123jkl -S 10.192.63.148  -i C:\SQLData\AWDimEmployees.csv -T AdventureWorksPDW2012.dbo.DimEmployees -R C:\SQLData\LoadErrors

B. Load Data into an AdventureWorks Table

The following example is part of a batch script that loads data into AdventureWorksPDW2012 . To view the full script, open the aw_create.bat file that ships with the AdventureWorksPDW2012 installation package. For more information, see Install AdventureWorksPDW2012 (SQL Server PDW).

The following script snippet uses dwloader to load data into the DimAccount and DimCurrency tables. This script is using an Ethernet address. If it was using InfiniBand, server would be <appliance_name>-SQLCTL01.

Copy image Copy Code
set server=10.193.63.134
set user=<MyUser>
set password=<MyPassword>

set schema=AdventureWorksPDW2012.dbo
set load="C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100\dwloader.exe"
set mode=reload

--Loads data into the AdventureWorksPDW2012.dbo.DimAccount table
--Source data is stored in the file DimAccount.txt,
--which is in the current directory.

set t1=DimAccount
%load% -S %server% -E -M %mode% -e Utf16 -i .\%t1%.txt -T %schema%.%t1% -R %t1%.bad -t "|" -r \r\n -U %user% -P %password%

--Loads data from the DimCurrency.txt file into
--AdventureWorksPDW2012.dbo.DimCurrency
set t1=DimCurrency
%load% -S %server% -E -M %mode% -e Utf16 -i .\%t1%.txt -T %schema%.%t1% -R %t1%.bad -t "|" -r \r\n -U %user% -P %password%

The following is the DDL for the DimAccount Table.

Copy image Copy Code
CREATE TABLE DimAccount(
AccountKey int NOT NULL,
ParentAccountKey int,
AccountCodeAlternateKey int,
ParentAccountCodeAlternateKey int,
AccountDescription nvarchar(50),
AccountType nvarchar(50),
Operator nvarchar(50),
CustomMembers nvarchar(300),
ValueType nvarchar(50),
CustomMemberOptions nvarchar(200))
with (CLUSTERED INDEX(AccountKey),
DISTRIBUTION = REPLICATE);

The following is an example of the data file, DimAccount.txt, that contains data to load into the table DimAccount.

Copy image Copy Code
--Sample of data in the DimAccount.txt load file.

1||1||Balance Sheet||~||Currency|
2|1|10|1|Assets|Assets|+||Currency|
3|2|110|10|Current Assets|Assets|+||Currency|
4|3|1110|110|Cash|Assets|+||Currency|
5|3|1120|110|Receivables|Assets|+||Currency|
6|5|1130|1120|Trade Receivables|Assets|+||Currency|
7|5|1140|1120|Other Receivables|Assets|+||Currency|
8|3|1150|110|Allowance for Bad Debt|Assets|+||Currency|
9|3|1160|110|Inventory|Assets|+||Currency|
10|9|1162|1160|Raw Materials|Assets|+||Currency|
11|9|1164|1160|Work in Process|Assets|+||Currency|
12|9|1166|1160|Finished Goods|Assets|+||Currency|
13|3|1170|110|Deferred Taxes|Assets|+||Currency|

C. Load Data from the Command Line

The script in Example B can be replaced by entering all the parameters on the command line, as shown in the following example.

Copy image Copy Code
C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100\dwloader.exe –S <Control node IP> -E –M reload –e UTF16 -i .\DimAccount.txt –T AdventureWorksPDW2012.dbo.DimAccount –R DimAccount.bad –t "|" –r \r\n –U <login> -P <password>

Description of the command-line parameters:

  • C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100\dwloader.exe is the installed location of dwloader.exe.

  • -S is followed by the IP address of the Control node.

  • -E specifies to load empty strings as NULL.

  • -M reload specifies to truncate the destination table before it inserts the source data.

  • -e UTF16 indicates the source file uses the little endian character encoding type.

  • -i .\DimAccount.txt specifies the data is in a file called DimAccount.txt which exists in the current directory.

  • -T AdventureWorksPDW2012.dbo.DimAccount specifies the 3-part name of the table to receive the data.

  • -R DimAccount.bad specifies the rows that fail to load will be written to a file called DimAccount.bad.

  • –t "|" indicates the fields in the input file, DimAccount.txt, are separated with the pipe character.

  • -r \r\n specifies each row in DimAccount.txt ends with a carriage return and a line feed character.

  • -U <login_name> -P <password> specifies the login and password for the login that has permissions to perform the load.

See Also


Monitor Loads (SQL Server PDW)

[image]

Monitor Loads (SQL Server PDW)
See Also Send Feedback

Active and recent dwloader loads can be monitored by using either the Monitor the Appliance by Using the Admin Console (Analytics Platform System) or the System Views (SQL Server PDW). See dwloader Command-Line Loader (SQL Server PDW) for information on the dwloader tool.

Tip Tip

Some loads are initiated by using INSERT statements or business intelligence tools that use SQL statements to perform the load. To monitor this type of load, see Monitoring Active Queries (SQL Server PDW).

Prerequisites

Regardless of the method used to monitor a load, the login must have permission to access the underlying data sources. For the permissions to grant, see “Use All of the Admin Console” in Grant Permissions to Use the Admin Console (SQL Server PDW).

Monitoring Loads

The following sections describe how to monitor loads.

To monitor loads by using the Admin Console

  1. Log on to the Admin Console. See Monitor the Appliance by Using the Admin Console (Analytics Platform System) for instructions.

  2. On the top menu, click Loads. You will see a sortable table showing all recent and active loads plus additional information, such as whether the load has completed or is still active. Click the column headers to sort the rows.

  3. To view additional details for a specific load, click the load ID in the left column. In the detailed view, you can see progress on each step of the load.

See these system views for information on the metadata about the load that is shown in the Admin Console:

  • sys.dm_pdw_exec_requests (SQL Server PDW)

  • sys.pdw_loader_run_stages (SQL Server PDW)

  • sys.pdw_loader_backup_runs (SQL Server PDW)

  • sys.pdw_loader_backup_run_details (SQL Server PDW)

To monitor loads by using system views

To monitor active and recent loads by using SQL Server PDW views, follow the steps below. For each system view used, see the documentation for that view for information on the columns and potential values returned by the view.

  1. Find the request_id for the load in the sys.dm_pdw_exec_requests (SQL Server PDW) view by finding the loader command line in the command column for this view.

    For example, the following command returns the command text and current status, plus the request_id.

    Copy image Copy Code
    SELECT request_id, status, command FROM sys.dm_pdw_exec_requests;

  2. Use the request_id to retrieve additional information for the load by using the sys.pdw_loader_run_stages (SQL Server PDW), sys.pdw_loader_backup_runs (SQL Server PDW), and sys.pdw_loader_backup_run_details (SQL Server PDW) views. For example, the following query returns the run_id and information on the start, end, and duration times of the load, plus any errors, and information on the number of rows processed:

    Copy image Copy Code
    SELECT lbr.run_id,
er.submit_time, er.end_time, er.total_elapsed_time, er.error_id, lbr.rows_processed, lbr.rows_rejected, lbr.rows_inserted
FROM sys.dm_pdw_exec_requests er
LEFT OUTER JOIN
sys.pdw_loader_backup_runs lbr
ON (er.request_id=lbr.requst_id)
WHERE er.request_id=’12738’;

See Also


Database Administration (SQL Server PDW)

[image]

Database Administration (SQL Server PDW)
See Also Send Feedback

These topics describe how database administrators can use the SQL Server PDW tools and processes to manage logins, server roles, and permissions, and to monitor the appliance health.

Database Administration Topics

  • Backup and Restore (SQL Server PDW)

  • PDW Permissions (SQL Server PDW)

  • Error Messages (SQL Server PDW)

  • Locking Behavior (SQL Server PDW)

  • PDW Domain Security (SQL Server PDW)

  • Remote Table Copy (SQL Server PDW)

  • System Databases (SQL Server PDW)

  • User Sessions (SQL Server PDW)

  • Workload Management (SQL Server PDW)

See Also


Backup and Restore (SQL Server PDW)

[image]

Backup and Restore (SQL Server PDW)
Send Feedback

Describes how data backup and restore works for SQL Server PDW. Backup and restore operations are used for disaster recovery. Backup and restore can also be used to copy a database from one appliance to another appliance.

Contents

  1. Backup and Restore Basics

  2. Backup Types

  3. Backup Process

  4. Restore Modes

  5. Restore Process

Backup and Restore Basics

A SQL Server PDW database backup is a copy of an appliance database, stored in a format so that it can be used to restore the original database to an appliance.

A SQL Server PDW database backup is created with the BACKUP DATABASE (SQL Server PDW) statement and formatted for use with the RESTORE DATABASE (SQL Server PDW) statement; it is unusable for any other purpose. The backup can only be restored to an appliance with the same number or a greater number of Compute nodes.

The master Database (SQL Server PDW) is a SMP SQL Server database. It is backed up with the BACKUP DATABASE statement, and restored by using the Restore the Master Database (Analytics Platform System) page of the Configuration Manager tool.

SQL Server PDW uses SQL Server backup technology to backup and restore appliance databases. SQL Server backup options are preconfigured to use backup compression. You cannot set backup options such as compression, checksum, block size, and buffer count.

Database backups are stored on one or more Backup servers, which exist in your own customer network. SQL Server PDW writes a user database backup in parallel directly from the Compute nodes to one backup server and restores a user database backup in parallel directly from the backup server to the Compute nodes.

Backups are stored on the backup server as a set of files in the Windows file system. A SQL Server PDW database backup can only be restored to SQL Server PDW. However, you can archive database backups from the backup server to another location by using standard Windows file backup processes. For more information about backup servers, see Acquire and Configure a Backup Server (SQL Server PDW).

[TOP]

Backup Types

There are two types of data that require a backup: user databases and system databases (e.g., the master database). A transaction log backup is not supported in this release.

A full database backup is a backup of an entire SQL Server PDW database. This is the default backup type. A backup of a user database includes database users, and database roles. This is the default backup type. A backup of master includes logins.

A differential backup contains all of the changes since the last full backup. A differential backup usually takes less time than a full backup and can be performed more frequently. When multiple differential backups are based on the same full backup, each differential includes all of the changes in the previous differential.

For example, you could create a full backup weekly and a differential backup daily. To restore the user database, the full backup plus the last differential (if one exists) needs to be restored.

A differential backup is only supported for user databases. A backup of master is always a full backup.

To backup the entire appliance, you need to perform a backup of all user databases and a backup of the master database.

[TOP]

Backup Process

The following diagram shows the flow of data during a database backup.

SQL Server PDW Backup Process

The backup process works as follows:

  1. User submits a BACKUP DATABASE SQL statement to the Control node.

    • The backup is either a full or differential backup.

  2. For user databases, the Control node (MPP Engine) creates a distributed query plan to perform a parallel database backup.

  3. Each node involved in the backup copies its backup file to the Backup server using SQL Server backup functionality.

    • Each node involved copies one backup file to the Backup server.

    • The user database backup (full or differential) includes a backup of the portion of the database stored on each Compute node, and a backup of the database users and database roles.

  4. The appliance performs the backup in parallel using the InfiniBand network.

    • SQL Server PDW performs each full and differential backup in parallel. However, multiple database backups do not run concurrently. Each backup request must wait for previously submitted backups to finish before it is run.

    • A backup of the master database only backs up data from the Control node. This backup type is performed serially.

  5. A SQL Server PDW database backup is a group of files stored in a directory that resides off the appliance. The directory name is specified as a network path and directory name. The directory cannot be a local path, and it cannot be on the appliance.

  6. After the backup is finished, you can use the Windows file system to copy the backup directory to another location, if desired.

    • A backup can only be restored to a SQL Server PDW appliance that has an equal or greater number of Compute nodes.

    • You cannot change the name of the backup before performing a restore. The name of the backup directory must match the name of the original name of the backup. The original name of the backup is located in the backup.xml file within the backup directory. To restore a database to a different name, you can specify the new name in the restore command. For example: RESTORE DATABASE MyDB1 FROM DISK = ꞌ\\10.192.10.10\backups\MyDB2ꞌ.

[TOP]

Restore Modes

A full database restore re-creates the SQL Server PDW database by using the data in the database backup. The database restore is performed by first restoring a full backup, and then optionally restoring one differential backup. The database restore includes the database users and database roles.

A header only restore returns the header information for a database. It does not restore data to the appliance.

An appliance restore is a restore of the entire appliance. This include a restore of all user databases and the master database.

[TOP]

Restore Process

The following diagram shows the flow of data during a database restore.

Restoring to an Appliance with the Same Number of Compute Nodes

When restoring data, the appliance detects the number of Compute nodes on the source appliance and the destination appliance. If both appliances have an equal number of Compute nodes, the restore process works as follows:

  1. The database backup to be restored is available on a Windows file share on a non-appliance backup server. For best performance, this server is usually connected to the appliance InfiniBand network.

  2. User submits a RESTORE DATABASE (SQL Server PDW) SQL statement to the SQL Server PDW Control node.

    • The restore is either a full restore or a header restore. The full restore restores a full backup and then optionally restores a differential backup.

  3. The Control node (MPP Engine) creates a distributed query plan to perform a parallel database restore.

    • SQL Server PDW performs the restore of a user database in parallel. However, multiple database backups and restores are not run concurrently. The MPP Engine puts each restore statement into a queue; it must wait for previously submitted backup and restore requests to finish before it is run.

    • A restore of the master database only restores data to the Control node; the restore is performed serially.

    • A restore of the header information is a quick operation and does not restore any data to the Compute or Control nodes. Instead, the Control node returns the results as query output.

  4. The backup files get copied to the correct Compute nodes in parallel, usually over the appliance InfiniBand network.

  5. Each Compute node restores its portion of the user database. If any of the restores do not finish successfully, all of the databases get removed and the restore completes unsuccessfully.

Restoring to an Appliance With a Larger Number of Compute Nodes

Restoring a backup to an appliance with a larger number of Compute nodes grows the allocated database size in proportion to the number of Compute nodes.

For example, when restoring a 60 GB database from a 2-node appliance (30 GB per node) to a 6-node appliance, SQL Server PDW creates a 180 GB database (6 nodes with 30 GB per node) on the 6-node appliance. SQL Server PDW initially restores the database to 2 nodes to match the source configuration, and then redistributes the data to all 6 nodes.

After the redistribution each Compute node will contain less actual data and more free space than each Compute node on the smaller source appliance. Use the additional space to add more data to the database. If the restored database size is larger than you need, you can use ALTER DATABASE (SQL Server PDW) to shrink the database file sizes.

[TOP]

Related Tasks

Backup and Restore Task | Description
Prepare a server as a backup server. | Acquire and Configure a Backup Server (SQL Server PDW)
Create a disaster recovery plan. | Create a Disaster Recovery Plan (SQL Server PDW)
Backup a database. | BACKUP DATABASE (SQL Server PDW)
Restore a database. | RESTORE DATABASE (SQL Server PDW)
Restore the master database. | To restore the master database, use the Restore the Master Database (Analytics Platform System) page in the Configuration Manager tool.
Copy a database from one appliance to another appliance. | Copy a SQL Server PDW Database to Another Appliance (SQL Server PDW).
Monitor Backups and Restores. | Monitor Backups and Restores (SQL Server PDW)


Restore a Database Protected by TDE (SQL Server PDW)

[image]

Restore a Database Protected by TDE (SQL Server PDW)
See Also Send Feedback

Use the following steps to restore a database that is encrypted by using transparent data encryption.

The Using Transparent Data Encryption section of the topic Transparent Data Encryption (SQL Server PDW) contains example code to enable TDE on the AdventureWorksPDW2012 database. The following code continues that example, by creating a backup of the database on the original SQL Server PDW, and then restoring the certificate and the database on a new SQL Server PDW.

The first step is to create a backup of the source database.

Copy image Copy Code
BACKUP DATABASE AdventureWorksPDW2012
TO DISK = '\\SECURE_SERVER\Backups\AdventureWorksPDW2012';

Prepare the new SQL Server PDW for TDE by creating a master key, enabling encryption, and creating a network credential.

Copy image Copy Code
USE master;
GO

-- Create a database master key in the master database
CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<UseStrongPasswordHere>';
GO

-- Enable encryption for PDW
EXEC sp_pdw_database_encryption 1;
GO

EXEC sp_pdw_add_network_credentials 'SECURE_SERVER', '<domain>\<Windows_user>', '<password>';

The last two steps recreate the certificate by using the backups from the original SQL Server PDW. Use the password that you used when you created the backup of the certificate.

Copy image Copy Code
-- Create certificate in master
CREATE CERTIFICATE MyServerCert
    FROM FILE = '\\SECURE_SERVER\cert\MyServerCert.cer'
    WITH PRIVATE KEY (FILE = '\\SECURE_SERVER\cert\MyServerCert.key',
    DECRYPTION BY PASSWORD = '<password>');

RESTORE DATABASE AdventureWorksPDW2012
    FROM DISK = '\\SECURE_SERVER\Backups\AdventureWorksPDW2012';

See Also

Reference

BACKUP DATABASE (SQL Server PDW)
CREATE MASTER KEY (SQL Server PDW)
sp_pdw_add_network_credentials (SQL Server PDW)
sp_pdw_database_encryption (SQL Server PDW)
CREATE CERTIFICATE (SQL Server PDW)
RESTORE DATABASE (SQL Server PDW)


Create a Disaster Recovery Plan (SQL Server PDW)

[image]

Create a Disaster Recovery Plan (SQL Server PDW)
See Also Send Feedback

A disaster recovery plan is a crucial part of managing a data warehouse. These best practices describe important aspects of a disaster recovery plan.

Disaster Recovery Checklist

The following checklist describes best practices to consider when implementing a disaster recovery plan.

Best Practice

Description

Schedule regular backups

Depending on your business requirements, backups should be scheduled regularly as part of your disaster recovery plan.

Use differential backups for frequent backups

Differential backups can be scheduled more frequently than full backups. For example, you could schedule a weekly full backup and a daily differential backup. If you retain all of the differentials after the last full backup, you can restore a database to any day after the last full backup.

Hence, if you create a full backup on Saturday, and the data gets corrupted on Thursday, you can restore the database by using the full backup from Saturday and the differential from Wednesday.

Back up all user databases and the appliance metadata

To restore an appliance, you will need the appliance metadata and all of the user databases.

Review the backup logs

Review the backup logs regularly to make sure the backups complete successfully.

Restore data regularly

Restore data regularly to verify that the backup and restore process is working end-to-end. To test the restore of an appliance database, you can restore the database to a test appliance or to a database with a different name on the production appliance. There must be enough storage space available to perform the test restore.

Copy a SQL Server PDW Database to Another Appliance (SQL Server PDW)

Use different logins and personnel for backup and restore

For security of the data, the logins and the people who are authorized to perform backups should not also be authorized to perform restores. This helps to keep the backup operator from unintentionally restoring a database to a production system.

Train multiple people how to do backup and restores

When a backup or restore is needed, the person who performs these operations on a regular basis might be unavailable. To help ensure that a qualified person is available, train more than one person how to perform the backup and restore tasks.

Store backup files in multiple locations

The disaster recovery plan should include storing backup files in more than one location. If disk space allows, leave backup files on the Backup server for fast restores, and copy them to another server in case of data loss on the Backup server. Disaster recovery plans should also include offsite storage depending upon business requirements.

Store backup devices and backup media in a secure location

Secure the access to the backup devices and media to minimize risk of the data getting stolen or corrupted.

[TOP]

See Also

Reference

BACKUP DATABASE (SQL Server PDW)
RESTORE DATABASE (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Backup and Restore (SQL Server PDW)


Copy a SQL Server PDW Database to Another Appliance (SQL Server PDW)

[image]

Copy a SQL Server PDW Database to Another Appliance (SQL Server PDW)
See Also Send Feedback

Use this process to copy a SQL Server PDW database from a source appliance to a destination appliance.

Before You Begin

Prerequisites

Verify that the destination appliance has the same number or more Compute nodes than the source appliance.

Perform the Copy

To copy a SQL Server PDW database to another appliance

  1. Use the BACKUP DATABASE (SQL Server PDW) statement to perform a backup of the database you want to copy. The backup must contain a full backup. A differential backup in addition to the full backup is optional.

  2. Locate the full and differential backup files. They are each stored in separate directories on the Backup server under G:\Backups. The name of the directories is determined by the parameters you enter in the Backup statement.

  3. Copy the backup directory or directories from G:\Backups on the Backup server of the source appliance to G:\Backups on the Backup server of the destination appliance.

    • To perform the copy, use the Windows file system to copy the data across your corporate network. Additionally, you can install your own software on the Backup server to assist with copying data across the corporate network.

    • You will get an error if the destination directories already exist.

    • You can change the name of each backup directory when you copy it to the destination appliance. For example, you could copy G:\Backups\MyFullBackup on the source appliance to G:\Backups\MyFullBackupToRestore on the destination appliance.

  4. Use the RESTORE DATABASE (SQL Server PDW) statement to restore the database to the destination appliance.

    • You can specify the name for the restored database when you run the RESTORE DATABASE statement.

    • The database to restore cannot already exist on the destination appliance.

See Also


Monitor Backups and Restores (SQL Server PDW)

[image]

Monitor Backups and Restores (SQL Server PDW)
See Also Send Feedback

Active and recent backups and restores can be monitored by using either the Monitor the Appliance by Using the Admin Console (Analytics Platform System) or the System Views (SQL Server PDW). See Backup and Restore (SQL Server PDW) for information on how to use backup and restore on SQL Server PDW.

Prerequisites

Regardless of the method used to monitor a backup or restore, the login must have permission to access the underlying data sources. See “Use All of the Admin Console” in Grant Permissions to Use the Admin Console (SQL Server PDW) for the permissions to grant.

Monitoring Backups and Restores

The following sections describe how to monitor both backups and restores.

To monitor backups and restores by using the Admin Console

  1. Log on to the Admin Console. See Monitor the Appliance by Using the Admin Console (Analytics Platform System) for instructions.

  2. On the top menu, click Backups/Restores. You will see a sortable table showing all recent and active backups and restores plus additional information, such as whether the action has completed or is still active. Click the column headers to sort the rows.

  3. To view additional details for a specific backup or restore, click ID in the left column. In the detailed view, you can see progress on each step of the backup or restore.

See these system views for information on the metadata about the backup or restore that is shown in the Admin Console:

  • sys.dm_pdw_exec_requests (SQL Server PDW)

  • sys.pdw_loader_run_stages (SQL Server PDW)

  • sys.pdw_loader_backup_runs (SQL Server PDW)

  • sys.pdw_loader_backup_run_details (SQL Server PDW)

To monitor backups and restores by using system views

For each system view used, see the documentation for that view for information on the columns and potential values returned by the view.

  1. Find the request_id for the backup or restore in the sys.dm_pdw_exec_requests (SQL Server PDW) view by finding the backup or restore command in the command column for this view.

    For example,

    Copy image Copy Code
    SELECT request_id, status, command FROM sys.dm_pdw_exec_requests;

    The preceding command returns the command text and the current status, plus request_id.

  2. Use the request_id to retrieve additional information for the backup or restore, using the sys.pdw_loader_run_stages (SQL Server PDW), sys.pdw_loader_backup_runs (SQL Server PDW), and sys.pdw_loader_backup_run_details (SQL Server PDW) views. For example, this query returns the run_id and information on the start, end, and duration times of the backup or restore, plus any errors, and information on the number of rows processed:

    Copy image Copy Code
    SELECT lbr.run_id,
er.submit_time, er.end_time, er.total_elapsed_time, er.error_id, lbr.rows_processed
FROM sys.dm_pdw_exec_requests er
LEFT OUTER JOIN
sys.pdw_loader_backup_runs lbr
ON (er.request_id=lbr.requst_id)
WHERE er.request_id=’12738’;

See Also


PDW Permissions (SQL Server PDW)

[image]

PDW Permissions (SQL Server PDW)
See Also Send Feedback

This topic describes the requirements and options for managing database permissions for SQL Server PDW.

Contents

  • Database Permission Basics

  • Default Permissions

  • Determining Permissions

  • Database Permissions Best Practices

Database Engine Permission Basics

Database Engine permissions on SQL Server PDW are managed at the server level through logins, and at the database level through database users and user-defined database roles.

Logins

Logins are individual user accounts for logging on to the SQL Server PDW. SQL Server PDW supports logins using Windows Authentication and SQL Server authentication. Windows Authentication logins can be Windows users or Windows groups from any domain that is trusted by SQL Server PDW. SQL Server Authentication logins are defined and authenticated by SQL Server PDW and must be created by specifying a password.

Members of the sysadmin fixed server role (such as the sa login) can connect to a database without having being mapped to a database user. They are mapped to the dbo user. The owner of the database is also mapped as the dbo user.

Server Roles

There are four special server roles with a set of preconfigured roles that provide convenient group of server-level permissions. The sysadmin, MediumRC, LargeRC, and XLargeRCfixed server roles are the only server roles currently implemented in SQL Server PDW. The sa login is the only member of the sysadmin fixed server role, and additional logins cannot be added to the sysadmin role. Logins can be granted the CONTROL SERVER permission, which is similar, though not identical, to the sysadmin fixed server role. Use ALTER SERVER ROLE (SQL Server PDW) to add members to the other server roles. SQL Server PDW does not support user-defined server roles.

Database Users

Logins are granted access to a database by creating a database user in a database and mapping that database user to a login. Typically the database user name is the same as the login name, though it does not have to be the same. Each database user maps to a single login. A login can be mapped to only one user in a database, but can be mapped as a database user in several different databases.

Fixed Database Roles

Fixed database roles are a set of preconfigured roles that provide convenient group of database-level permissions. Database users and user-defined database roles can be added to the fixed database roles using the sp_addrolemember (SQL Server PDW) procedure. For more information about fixed database roles, see Fixed Database Roles (SQL Server PDW).

User-defined Database Roles

Users with the CREATE ROLE permission can create new user-defined database roles to represent groups of users with common permissions. Typically permissions are granted or denied to the entire role, simplifying permissions management and monitoring.

Permissions are granted to security principals (logins, users, and roles) by using the GRANT statement. Permissions are explicitly denied by using the DENY command. A previously granted or denied permission is removed by using the REVOKE statement. Permissions are cumulative, with the user receiving all the permissions granted to the user, login, and any group memberships; however any permission denial overrides all grants. (For information, syntax, and available permissions with these commands, see Permissions: GRANT, DENY, REVOKE (SQL Server PDW)).

The following example represents a common and recommended method of configuring permissions.

  1. If using Windows Authentication, create a login for each Windows user or Windows group who will connect to SQL Server PDW. If using SQL Server Authentication, create a login for each person who will connect to SQL Server PDW.

  2. Create a database user for each login in all necessary databases.

  3. Create one or more user-defined database roles, each representing a similar function. For example financial analyst, and sales analyst.

  4. Add database users to one or more user-defined database roles.

  5. Grant permissions to the user-defined database roles.

Logins are server-level objects and can be listed by viewing sys.server_principals. Only server-level permissions can be granted to server principals.

Users and database roles are database-level objects and can be listed by viewing sys.database_principals. Only database-level permissions can be granted to database principals.

[TOP]

Default Permissions

The following list describes the default permissions:

  • When a login is created by usings CREATE LOGIN statement, the login receives the CONNECT SQL permission allowing the login to connect to the SQL Server PDW.

  • When a database user is created by using the CREATE USER statement, the user receives the CONNECT ON DATABASE::<database_name> permission, allowing the login to connect to that database as a user.

  • All principals, including the PUBLIC role, have no explicit or implicit permissions by default because implicit permissions are inherited from explicit permissions. Therefore, when no explicit permissions are present, there can also be no implicit permissions.

  • When a login becomes the owner of an object or database, the login always has all permissions on the object or database. The ownership permissions are not visible as explicit permissions. The GRANT, REVOKE, and DENY statements have no effect on ownership permissions. Ownership can be changed by using the ALTER AUTHORIZATION (SQL Server PDW) statement.

  • The sa login has all permissions on the appliance. Similar to ownership permissions, the sa permissions cannot be changed and are not visible as explicit permissions. The GRANT, REVOKE, and DENY statements have no effect on sa permissions.

  • The PUBLIC server role receives no permissions by default and does not inherit permissions from other server roles. The PUBLIC server role can be given explicit permissions with the GRANT, REVOKE, and DENY statements.

  • Transactions do not require permissions. All principals can run the BEGIN TRANSACTION, COMMIT, and ROLLBACK transaction commands. However, a principal must have the appropriate permissions to run each statement within the transaction.

  • The USE statement does not require permissions. All principals can run the USE statement on any database, however to access a database they must have a user principal in the database or the guest user must be enabled.

The PUBLIC Role

All new appliance logins automatically belong to the PUBLIC role. The PUBLIC server role has the following characteristics:

  • The PUBLIC server role has no permissions by default.

  • All principals are members of the PUBLIC server role, and the PUBLIC server role is not a member of another server role.

  • The PUBLIC server role cannot inherit implicit permissions. Any permissions given to the PUBLIC role must be explicitly granted.

[TOP]

Determining Permissions

Whether or not a login has permission to perform a specific action depends on the permissions granted or denied to login, user, and roles the user is a member of. Server-level permissions (such as CREATE LOGIN and VIEW SERVER STATE) are available to server-level principals (logins). Database-level permissions (such as SELECT from a table or EXECUTE on a procedure) are available to database-level principals (users and database roles).

Implicit and Explicit Permissions

An explicit permission is a GRANT or DENY permission given to a principal by a GRANT or DENY statement. Database-level permissions are listed in the sys.database_permissions (SQL Server PDW) view. Server-level permissions are listed in the sys.server_permissions (SQL Server PDW) view.

An implicit permission is a GRANT or DENY permission that a principal (login or server role) has inherited. A permission can be inherited in the following ways.

  • A principal can inherit a permission from a role if the principal is a member of the role even if the principal does not have an explicit GRANT or DENY permission.

  • A principal can inherit a permission on a subordinate object (such as a table) if the principal has a permission on one of the objects parent scopes (such as the schema of the table or the permission on the entire database).

  • A principal can inherit a permission by having a permission that includes a subordinate permission. For example the ALTER ANY USER permission includes the both the CREATE USER and the ALTER ON USER::<name> permissions.

Determining Permissions When Performing Actions

The process of determining which permission to assign to a principal is complex. The complexity occurs when determining implicit permissions because principals can be members of multiple roles and permissions can be passed through multiple levels in the role hierarchy.

The following list describes general rules for determining permissions:

  • Ownership implies permission.

    An object owner has all permissions on the object. Likewise, a database owner has all permissions on the database and all permissions on the objects in the database. These permissions cannot be changed.

  • Permissions can be inherited across multiple levels in the hierarchy of server role memberships.

    For example, suppose you have the following situation:

    • Login David is a member of database role PerfAnalysts.

    • PerfAnalysts is a member of database role Production.

    • David and PerfAnalysts have no SELECT permission on the Customer table. The permission was either revoked or never explicitly granted.

    • Production has SELECT permission on the Customer table.

    In this case, PerfAnalysts will inherit GRANT permission on the Customer table from Production, and David will inherit GRANT permission on the Customer table from Production.

  • DENY overrides GRANT when permissions conflict.

    For example, suppose login David has no permissions on the Customer table and is a member of two database roles –dbgroup1, which has DENY permission on the Customer table, and dbgroup2, which has GRANT permission on the Customer table. In this case, David will inherit the DENY permission on the Customer table. This is the case whether the roles gained their permissions explicitly or implicitly.

[TOP]

Auditing Permissions

To research the permissions of a user check the following.

  • Execute the following query to determine which logins are system administrators.

    Copy image Copy Code
    SELECT SPLogins.name, 'is a member of ', SPRoles.name
FROM sys.server_role_members AS SRM
JOIN sys.server_principals AS SPRoles
    ON SRM.role_principal_id = SPRoles.principal_id
JOIN sys.server_principals AS SPLogins
    ON SRM.member_principal_id = SPLogins.principal_id;

  • Execute the following query to determine which logins have been granted explicit permissions.

  • Copy image Copy Code
    SELECT name, 'has the ', state_desc , permission_name, ' permission'
FROM sys.server_permissions AS SP
JOIN sys.server_principals AS SPRoles
    ON SP.grantee_principal_id = SPRoles.principal_id;

  • Execute the following query in a user database to determine which database users are members of a database role.

    Copy image Copy Code
    SELECT DPUsers.name, 'is a member of ', DPRoles.name
FROM sys.database_role_members AS DRM
JOIN sys.database_principals AS DPRoles
    ON DRM.role_principal_id = DPRoles.principal_id
JOIN sys.database_principals AS DPUsers
    ON DRM.member_principal_id = DPUsers.principal_id;

  • Execute the following query in a user database to determine which database users and roles have been granted or denied specific permissions. You will have to query additional views such as sys.objects and sys.schemas to identify the items described with the major_id.

    Copy image Copy Code
    SELECT DPUsers.name, 'has the ', permission_name,
'permission on the item described as class = ', class, 'id = ', major_id
FROM sys.database_permissions AS DP
JOIN sys.database_principals AS DPUsers
    ON DP.grantee_principal_id = DPUsers.principal_id;

Database Permissions Best Practices

  • Grant permissions at the most granular level that is practical. Granting permissions at the table or view level permissions could become unmanageable. But granting permissions at the database level could be too permissive. If the database is designed with schemas to define work boundaries, perhaps the granting permission to the schema is an appropriate compromise between the table level and the database level.

  • Grant permissions to roles, rather than to users or logins. Managing rights by using roles instead of users makes it easy to quickly grant or revoke a set of permissions for a user or login by moving them into or out of the role. When a function passes from one person to another, the permissions can remain intact at the role level while the role membership changes.

  • Grant permissions to roles based on job function, and on higher-level group roles that combine the job function roles based on the company group performing the actions.

  • Every end user should have a unique login. Do not allow users to share logins. Providing a login for each user ensures an audit trail and simplifies permission management.

[TOP]

Related Topics

  • Fixed Server Roles (SQL Server PDW)

  • Fixed Database Roles (SQL Server PDW)

  • Grant Permissions to Submit Database Queries (SQL Server PDW)

  • Grant Permissions to Use the Admin Console (SQL Server PDW)

  • Grant Permissions to Load Data (SQL Server PDW)

  • Grant Permissions to Copy Data Off the Appliance (SQL Server PDW)

  • Grant Permissions to Manage Databases (SQL Server PDW)

  • Grant Permissions to Manage Logins, Users, and Database Roles (SQL Server PDW)

  • Grant Permissions to Monitor the Appliance (SQL Server PDW)

See Also


Fixed Server Roles (SQL Server PDW)

[image]

Fixed Server Roles (SQL Server PDW)
See Also Send Feedback

Fixed server roles are created automatically by SQL Server. SQL Server PDW has a limited implementation of SQL Server fixed server roles. Only the sysadmin and public have user logins as members. The setupadmin and dbcreator roles are used internally by SQL Server PDW. Additional members cannot be added or removed from any role.

sysadmin Fixed Server Role

Members of the sysadmin fixed server role can perform any activity in the server. The sa login is the only member of the sysadmin fixed server role. Additional logins cannot be added to the sysadmin fixed server role. Granting the CONTROL SERVER permission approximates membership in the sysadmin fixed server role. The following example grants the CONTROL SERVER permission to a login named Fay.

Copy image Copy Code
USE master;
GO
GRANT CONTROL SERVER TO Fay;
Security note Security Note

The CONTROL SERVER permission provides nearly complete control of SQL Server PDW. Whenever possible, provide more granular permissions to logins instead. For example, consider granting the VIEW SERVER STATE, ALTER ANY LOGIN, VIEW ANY DATABASE, or CREATE ANY DATABASE permissions.

public Server Role

Every login that can connect to SQL Server PDW is a member of the public server role. All logins inherit the permissions granted to public on any object. Only assign public permissions on an object when you want the object to be available to all users. You cannot change membership in the public role.

Note Note

public is implemented differently than other roles. Because all server principals are members of public, the membership of the public role is not listed in the sys.server_role_members DMV.

Fixed Server Roles vs. Granting Permissions

The system of fixed server roles and fixed database roles is a legacy system originated in the 1980's. Fixed roles are still supported and are useful in environments where there are few users and the security needs are simple. Beginning with SQL Server 2005, a more detailed system of granting permission was created. This new system is more granular, providing many more options for both granting and denying permissions. The extra complexity of the more granular system makes it harder to learn, but most enterprise systems should grant permissions instead of using the fixed roles. The permissions are discussed and listed in the topic Permissions: GRANT, DENY, REVOKE (SQL Server PDW).

See Also

Reference

Permissions: GRANT, DENY, REVOKE (SQL Server PDW)

Concepts

PDW Permissions (SQL Server PDW)


Fixed Database Roles (SQL Server PDW)

[image]

Fixed Database Roles (SQL Server PDW)
See Also Send Feedback

SQL Server provides pre-configured (fixed) database-level roles to help you manage the permissions on a server. The pre-configured roles are fixed in that you cannot change the permissions assigned to them. User-defined database roles can also be created. You can change the permissions assigned to user-defined database roles.

Roles are security principals that group other principals. Database roles are database-wide in their permissions scope. Database users and other database roles can be added as members of database roles. The fixed database roles cannot be added to each other. (Roles are like groups in the Windows operating system.)

Fixed Database Roles

There are 9 fixed database roles.

  • db_owner

  • db_securityadmin

  • db_accessadmin

  • db_backupoperator

  • db_ddladmin

  • db_datawriter

  • db_datareader

  • db_denydatawriter

  • db_denydatareader

Permissions of the Fixed Database Roles

The system of fixed server roles and fixed database roles is a legacy system originated in the 1980's. Fixed roles are still supported and are useful in environments where there are few users and the security needs are simple. Beginning with SQL Server 2005, a more detailed system of granting permission was created. This new system is more granular, providing many more options for both granting and denying permissions. The extra complexity of the more granular system makes it harder to learn, but most enterprise systems should grant permissions instead of using the fixed roles. The permissions are discussed and listed in the topic Permissions: GRANT, DENY, REVOKE (SQL Server PDW). The following chart shows the permissions that are associated with each fixed database role. All permissions in this SQL Server graphic are not available (or necessary) in APS.

APS security fixed database roles

Related Content

  • To create user-defined roles, see CREATE ROLE (SQL Server PDW).

See Also


Grant Permissions to Submit Database Queries (SQL Server PDW)

[image]

Grant Permissions to Submit Database Queries (SQL Server PDW)
See Also Send Feedback

This topic describes how to grant permissions to database roles and users to query data on the SQL Server PDW appliance.

Grant Permissions to Query Data

The statements used to grant permissions to query data depend on the scope of access desired. The following SQL statements create a login named KimAbercrombie that can access the appliance, create a database user named KimAbercrombie in the AdventureWorksPDW2012 database, create a database role named PDWQueryData, adds the use KimAbercrombie to the PDWQueryData role, and then show options for granting query access, based on whether the access is granted at the object, or database level.

Copy image Copy Code
USE master;
GO

CREATE LOGIN KimAbercrombie WITH PASSWORD = 'A2c3456$#' MUST_CHANGE,
CHECK_EXPIRATION = ON,
CHECK_POLICY = ON;
GO

USE AdventureWorksPDW2012;
GO

CREATE USER KimAbercrombie;
GO

CREATE ROLE PDWQueryData;
GO

EXEC sp_addrolemember 'PDWQueryData', 'KimAbercrombie'
GO

-- If the permission is granted against a table or view only, use this syntax:
GRANT SELECT ON OBJECT::AdventureWorksPDW2012..DimEmployee TO PDWQueryData;
GO

-- If the permission is granted for the database, use this syntax:
GRANT SELECT ON DATABASE::AdventureWorksPDW2012 TO PDWQueryData;
GO

-- If KimAbercrombie is the only user that needs to access a table, use this syntax:
GRANT SELECT ON OBJECT::AdventureWorksPDW2012..DimEmployee TO KimAbercrombie;
GO

See Also


Grant Permissions to Use the Admin Console (SQL Server PDW)

[image]

Grant Permissions to Use the Admin Console (SQL Server PDW)
See Also Send Feedback

This topic describes how to grant permissions to logins to use the Admin Console.

Grant Permissions to Monitor Appliance Health and Use the Admin Console

Use the Admin Console

To use the Admin Console a login requires the server level VIEW SERVER STATE permission. The following SQL statement grants the VIEW SERVER STATE permission to the login KimAbercrombie so that Kim can use the Admin Console to monitor the SQL Server PDW appliance.

Copy image Copy Code
USE master;
GO
GRANT VIEW SERVER STATE TO KimAbercrombie;
GO

Kill Sessions

To grant a login the permission to kill sessions, grant the ALTER ANY CONNECTION permission as follows:

Copy image Copy Code
GRANT ALTER ANY CONNECTION TO KimAbercrombie;

[TOP]

See Also


Grant Permissions to Load Data (SQL Server PDW)

[image]

Grant Permissions to Load Data (SQL Server PDW)
See Also Send Feedback

This topic describes how to grant permissions to database roles and database users to load data onto the SQL Server PDW appliance.

Grant Permissions to Load Data

The following script shows which permissions are required for each loading option. You can modify this to meet your specific needs.

Copy image Copy Code
-- Create server login for the examples that follow.
USE master;
CREATE LOGIN BI_ETLUser WITH PASSWORD = '******';

--Grant BULK Load permissions
GRANT ADMINISTER BULK OPERATIONS TO BI_ETLUser;

--Grant Staging database permissions
USE stagedb;
CREATE USER BI_ETLUser for login BI_ETLUser;
EXEC sp_addrolemember 'db_datareader','BI_ETLUser';
EXEC sp_addrolemember 'db_datawriter','BI_ETLUser';
EXEC sp_addrolemember 'db_ddladmin','BI_ETLUser';

-- The CREATE TABLE permission is required for the database hosting the temporary table.
-- This may be the staging database (if specified) or destination database.
-- The CREATE permission is not required if loading in fastappend mode.
GRANT CREATE TABLE ON database::stagedb TO BI_ETLUser;

-- If loading in append or fastappend mode, the INSERT permission is required on the destination table.
GRANT INSERT ON database::stagedb TO BI_ETLUser;

-- If loading in reload mode, the INSERT and DELETE permissions are required on the destination table.
GRANT INSERT ON database::stagedb TO BI_ETLUser;
GRANT DELETE ON database::stagedb TO BI_ETLUser;

-- If loading in upsert mode, the INSERT and UPDATE permissions are required on the destination table.
GRANT INSERT ON database::stagedb TO BI_ETLUser;
GRANT UPDATE ON database::stagedb TO BI_ETLUser;

-- Destination DB
USE tpch_1gb;
CREATE USER BI_ETLUser FOR LOGIN BI_ETLUser;
EXEC sp_addrolemember 'db_datareader','BI_ETLUser';
EXEC sp_addrolemember 'db_datawriter','BI_ETLUser';

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Load (SQL Server PDW)


Grant Permissions to Copy Data Off the Appliance (SQL Server PDW)

[image]

Grant Permissions to Copy Data Off the Appliance (SQL Server PDW)
See Also Send Feedback

This topic describes how to grant permissions to a user or database role to copy data off the SQL Server PDW appliance.

Grant Permissions to Copy Data Off the Appliance

To move data to another location requires SELECT permission on the table containing the data to be moved.

If the destination for the data is another SQL Server PDW, the user must have CREATE TABLE permission at the destination and ALTER SCHEMA permission on the schema that will contain the table.

See Also


Grant Permissions to Manage Databases (SQL Server PDW)

[image]

Grant Permissions to Manage Databases (SQL Server PDW)
See Also Send Feedback

This topic describes how to grant permissions to a database user to manage a database on the SQL Server PDW appliance.

Grant Permissions to Manage Databases

In some situations, a company assigns a manager for a database. The manager controls the access that other logins have to the database, as well as the data and objects in the database. To manage all objects, roles, and users in a database grant the user the CONTROL permission on the database. The following statement grants the CONTROL permission on the AdventureWorksPDW2012 database to the user KimAbercrombie.

Transact-SQL Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
GO
GRANT CONTROL ON DATABASE::
    AdventureWorksPDW2012
   TO KimAbercrombie;

To grant someone the permission to control all the databases on the appliance, grant the ALTER ANY DATABASE permission in the master database.

See Also


Grant Permissions to Manage Logins, Users, and Database Roles (SQL Server PDW)

[image]

Grant Permissions to Manage Logins, Users, and Database Roles (SQL Server PDW)
See Also Send Feedback

This topic describes how to grant permissions to manage logins, database users, and database roles.

Grant Permissions to Manage Logins

Add or Manage Logins

The following SQL statements create a Login named KimAbercrombie that can create new logins by using the CREATE LOGIN (SQL Server PDW) statement and alter existing logins by using the ALTER LOGIN (SQL Server PDW) statement.

The ALTER ANY LOGIN permission grants the ability to create new logins and drop exisiting. Once a login exists, the login can be managed by logins with the ALTER ANY LOGIN permission or the ALTER permission on that login. A login can change the password and default database for its own login.

Transact-SQL Copy image Copy Code
CREATE LOGIN KimAbercrombie
WITH PASSWORD = 'A2c3456$#' MUST_CHANGE,
CHECK_EXPIRATION = ON,
CHECK_POLICY = ON;
GO

GRANT ALTER ANY LOGIN TO KimAbercrombie;

Grant Permissions to Manage Login Sessions

To have the ability to view all sessions on the server requires the VIEW SERVER STATE permission. The ability to terminate the sessions of other logins requires the ALTER ANY CONNECTION permission. The following example uses the KimAbercrombie login created earlier.

Copy image Copy Code
-- Grant permissions to view sessions and queries
GRANT VIEW SERVER STATE TO KimAbercrombie;

-- Grant permission to end sessions
GRANT ALTER ANY CONNECTION TO KimAbercrombie;

Grant Permission to Manage Database Users

Creating and dropping database users requires the ALTER ANY USER permission. Managing existing users requires the ALTER ANY USER permission or the ALTER permission on that user. The following example uses the KimAbercrombie login created earlier.

Copy image Copy Code
-- Create a user
USE
    AdventureWorksPDW2012
  ;
GO
CREATE USER KimAbercrombie;

-- Grant permissions to create and drop users
GRANT ALTER ANY USER TO KimAbercrombie;

Grant Permisson to Manage Database Roles

Create and dropping user-defined database roles requires the ALTER ANY ROLE permission. The following example uses the KimAbercrombie login and use created earlier.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
GO
-- Grant permissions to create and drop roles
GRANT ALTER ANY ROLE TO KimAbercrombie;

Login, User, and Role Permission Charts

The following charts can be confusing, but they show how higher lever permissions (such as CONTROL) include more granular permissions that can be granted separately (such as ALTER). It is a best practice to always grant the least amount of permissions for someone to complete their necessary tasks. To do that, grant more specific permissions, instead of the top level permissions.

Login permissions:

APS security login permissions

User permissions:

APS security user permissions

Role permissions:

APS security role permissions

For a list of all permissions, see Permissions: GRANT, DENY, REVOKE (SQL Server PDW).

See Also

Reference

CREATE LOGIN (SQL Server PDW)
CREATE USER (SQL Server PDW)
CREATE ROLE (SQL Server PDW)
Permissions: GRANT, DENY, REVOKE (SQL Server PDW)


Grant Permissions to Monitor the Appliance (SQL Server PDW)

[image]

Grant Permissions to Monitor the Appliance (SQL Server PDW)
See Also Send Feedback

The SQL Server PDW appliance can be monitored by using either the Admin Console or SQL Server PDW system views. Logins require the server level VIEW SERVER STATE permission to monitor the appliance. Logins require the ALTER ANY CONNECTION permission to terminate connections by using the Admin Console or the KILL command. For information on permissions required to use the Admin Console, see Grant Permissions to Use the Admin Console (SQL Server PDW).

Grant Permission to Monitor the Appliance by Using System Views

The following SQL statements create a login named monitor_login and grants the VIEW SERVER STATE permission to the monitor_login login.

Copy image Copy Code
USE master;
GO
CREATE LOGIN monitor_login WITH PASSWORD='Password4321';
GRANT VIEW SERVER STATE TO monitor_login;
GO

Grant Permission to Monitor the Appliance by Using System Views and to Terminate Connections

The following SQL statements create a login named monitor_and_terminate_login and grants the VIEW SERVER STATE and ALTER ANY CONNECTION permissions to the monitor_and_terminate_login login.

Copy image Copy Code
USE master;
GO
CREATE LOGIN monitor_and_terminate_login WITH PASSWORD='Password1234';
GRANT VIEW SERVER STATE TO monitor_and_terminate_login;
GRANT ALTER ANY CONNECTION TO monitor_and_terminate_login;
GO

To create admin logins, see Fixed Server Roles (SQL Server PDW).

See Also

Reference

Permissions: GRANT, DENY, REVOKE (SQL Server PDW)


Error Messages (SQL Server PDW)

[image]

Error Messages (SQL Server PDW)
See Also Send Feedback

SQL Server PDW error messages report errors and problems encountered by the SQL Server PDW components and can also include SQL Server errors surfaced through SQL Server PDW. These error messages use a consistent syntax for presenting information. Understanding this syntax will allow you to identify and correct problems on SQL Server PDW.

Error Message Basics

Error messages that are returned follow the same syntax.

Copy image Copy Code
Error_Indicator [SQL_State_Code] [Driver_Details] [QueryID] Message_String

These are the potential values for each field:

Field | Description | Example
Error_Indicator | The word “ERROR” or other text alerting the user to a problem. | ERROR
SQL_State_Code | The SQL state code, according to ODBC specification. The driver generates the appropriate SQL State code any time it returns a message to an application. The text “Microsoft” indicates the source of the error. | 42000
Driver_Details | Driver-dependent details, such as the type of driver used. | ODBC SQL Server 2008 R2 Parallel Data Warehouse driver
QueryID | The unique identifier for the query. Use this value to find additional information related to processing of the query. For example, the query execution details can be found in the Admin Console using the query ID. For more information, see Monitor the Appliance by Using the Admin Console (Analytics Platform System). If a QueryID is not applicable, the text “Internal” is returned to the user. | QID2377
Message_String | A human-readable description of the error or problem. When returning SQL Server errors, this is the SQL Server message text. | Only equal assignment can appear in the set list of an UPDATE statement.

These example values would be presented to the user like this:

ERROR [42000] [Microsoft][ODBC SQL Server 2008 R2 Parallel Data Warehouse driver][QID2380]Only equal assignment can appear in the set list of an UPDATE statement.

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Understanding Admin Console Alerts (Analytics Platform System)


Locking Behavior (SQL Server PDW)

[image]

Locking Behavior (SQL Server PDW)
See Also Send Feedback

SQL Server PDW uses locking to ensure the integrity of transactions and to maintain the consistency of databases when multiple users are accessing data at the same time.

Contents

  • Locking Basics

  • General Remarks

Locking Basics

Modes

SQL Server PDW supports four locking modes:

Exclusive

The Exclusive lock prohibits writing to or reading from the locked object until the transaction holding the Exclusive lock completes. No other locks of any mode are permitted while the Exclusive lock is in effect. For example, DROP TABLE and CREATE DATABASE use an Exclusive lock.

Shared

The Shared lock prohibits initiation of an Exclusive lock on the affected object, but allows all other lock modes. For example, the SELECT statement initiates a Shared lock, and therefore allows multiple queries to access the selected data concurrently, but prevents updates to the records being read, until the SELECT statement completes.

ExclusiveUpdate

The ExclusiveUpdate lock prohibits writing to the locked object, but does allow reading via the Shared lock. No other locks are permitted while the ExclusiveUpdate lock is in effect. For example, BACKUP DATABASE and RESTORE DATABASE use an Exclusive Update lock.

SharedUpdate

The SharedUpdate lock prohibits Exclusive and ExclusiveUpdate lock modes and allows Shared and SharedUpdate lock modes on the object. SharedUpdate modifies an object, but does not restrict read access to it during the modification. For example, INSERT and UPDATE use a SharedUpdate lock.

Resource Classes

Locks are held on the following classes of objects: DATABASE, SCHEMA, OBJECT (a table, view, or procedure), APPLICATION (used internally), EXTERNALDATASOURCE, EXTERNALFILEFORMAT AND SCHEMARESOLUTION (a database level lock taken while creating, altering, or dropping schema objects or database users). These object classes can appear in the object_type column of sys.dm_pdw_waits.

[TOP]

General Remarks

Locks can be applied to databases, tables, or views.

SQL Server PDW does not implement any configurable isolation levels. It supports the READ_UNCOMMITTED isolation level as defined by the ANSI standard. However, since read operations are run under READ_UNCOMMITTED, very few blocking operations actually occur or lead to contention in the system.

SQL Server PDW relies on the underlying SQL Server engine to implement locking and concurrency control. If operations lead to an underlying SQL Server deadlock within the same node, SQL Server PDW leverages the SQL Server deadlock detection capability and terminates one of the blocking statements.

Note Note

SQL Server does not allow statements that are waiting for locks to be blocked by newer lock requests. SQL Server PDW has not fully implemented this process. In SQL Server PDW, continuous requests for new shared locks can sometimes block a previous (but waiting) request for an exclusive lock. For example, an UPDATE statement (requiring an exclusive lock) can be blocked by shared locks that are granted for series of SELECT statements. To resolve a blocked process (identified by reviewing the sys.dm_pdw_waits (SQL Server PDW) DVM), stop submitting new requests until the exclusive lock has been satisfied.

[TOP]

Lock Definition Table

SQL Server supports the following types of locks. Not all lock types are available on the control node, but could occur on the compute nodes.

  • Sch-S (Schema stability). Ensures that a schema element, such as a table or index, is not dropped while any session holds a schema stability lock on the schema element.

  • Sch-M (Schema modification). Must be held by any session that wants to change the schema of the specified resource. Ensures that no other sessions are referencing the indicated object.

  • S (Shared). The holding session is granted shared access to the resource.

  • U (Update). Indicates an update lock acquired on resources that may eventually be updated. It is used to prevent a common form of deadlock that occurs when multiple sessions lock resources for potential update in the future.

  • X (Exclusive). The holding session is granted exclusive access to the resource.

  • IS (Intent Shared). Indicates the intention to place S locks on some subordinate resource in the lock hierarchy.

  • IU (Intent Update). Indicates the intention to place U locks on some subordinate resource in the lock hierarchy.

  • IX (Intent Exclusive). Indicates the intention to place X locks on some subordinate resource in the lock hierarchy.

  • SIU (Shared Intent Update). Indicates shared access to a resource with the intent of acquiring update locks on subordinate resources in the lock hierarchy.

  • SIX (Shared Intent Exclusive). Indicates shared access to a resource with the intent of acquiring exclusive locks on subordinate resources in the lock hierarchy.

  • UIX (Update Intent Exclusive). Indicates an update lock hold on a resource with the intent of acquiring exclusive locks on subordinate resources in the lock hierarchy.

  • BU. Used by bulk operations.

  • RangeS_S (Shared Key-Range and Shared Resource lock). Indicates serializable range scan.

  • RangeS_U (Shared Key-Range and Update Resource lock). Indicates serializable update scan.

  • RangeI_N (Insert Key-Range and Null Resource lock). Used to test ranges before inserting a new key into an index.

  • RangeI_S. Key-Range Conversion lock, created by an overlap of RangeI_N and S locks.

  • RangeI_U. Key-Range Conversion lock, created by an overlap of RangeI_N and U locks.

  • RangeI_X. Key-Range Conversion lock, created by an overlap of RangeI_N and X locks.

  • RangeX_S. Key-Range Conversion lock, created by an overlap of RangeI_N and RangeS_S. locks.

  • RangeX_U. Key-Range Conversion lock, created by an overlap of RangeI_N and RangeS_U locks.

  • RangeX_X (Exclusive Key-Range and Exclusive Resource lock). This is a conversion lock used when updating a key in a range.

[TOP]

See Also

Reference

sys.dm_pdw_waits (SQL Server PDW)


PDW Domain Security (SQL Server PDW)

[image]

PDW Domain Security (SQL Server PDW)
Send Feedback

This topic summarizes the domain structure. Use this list to determine which component to connect to.

There are four layers where components can be configured.

  • Hardware – Physical adjustments to the CPU's and disks.

  • Appliance Domain/Region – The domain for all physical and virtual servers in the appliance. The Active Directory Domain Services, Windows Failover Clustering and Virtual Machine Manager are hosted in the fabric organizational unit (OU) of the appliance domain.

  • PDW Region – The Control and Compute VMs that belong to the PDWRegion organizational unit of the appliance domain.

  • HDInsight Region – The HDI Management, Secure, Head, and Data VMs that belong to the HDIRegion organizational unit of the appliance domain. For more information about the HDInsight region, see Understanding the Security Model of the HDInsight Region (Analytics Platform System).

Note Note

Domain names are provided during setup and are specified as up to 6 alphanumeric characters, starting with a letter. A frequent naming system creates an appliance domain starting with F, a PDW workload region starting with P, and an HDInsight region starting with H. This format is presumed throughout the help file topics but is not required.

Do the following activities using the Configuration Manager on the Control node

Use Remote Desktop to connect to the Control node ( PDW_region-CTL01) virtual machine, and then open C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100\dwconfig.exe.

  • Change the passwords for the appliance domain administrator and the PDW sa login. You can also use the ALTER LOGIN statement to change the sa password.

    Caution note Caution

    Analytics Platform System does not support the dollar sign character ($) in the domain administrator or local administrator passwords. A password containing a dollar sign will validate and be usable but can block upgrade and maintenance activities.

  • Set the time zone.

  • Set externally facing network setting for nodes.

  • Import and remove security certificates.

  • Configure firewall settings.

  • Set Instant File Initialization.

  • Restore the master database.

For more information about the Configuration Manager tool, see Appliance Configuration (Analytics Platform System).

Do the following activities using the Active Directory Users and Computers program (dsa.msc)

Use Remote Desktop to the Active Directory virtual machine (<appliance_domain>-AD01 or appliance_domain-AD02).

  • Create additional domain administrators in the appliance domain.

Do the following activities using the Admin Console

Using Internet Explorer, connect to the IP address of the Control node cluster (https://###.###.###.###/) specifying the login and password of a SQL Server PDW login. Logins that are not members of the sysadmin fixed server role can connect but may not be able to perform all activities.

  • Various tasks related to the SQL Server PDW health and performance.

Additional Domain and Security Topics

Security - Windows Authentication and SQL Server Authentication (SQL Server PDW)

Security - Configure Domain Trusts (SQL Server PDW)

PDW Permissions (SQL Server PDW)

Understanding the Security Model of the HDInsight Region (Analytics Platform System)


Security - Windows Authentication and SQL Server Authentication (SQL Server PDW)

[image]

Security - Windows Authentication and SQL Server Authentication (SQL Server PDW)
Send Feedback

SQL Server PDW supports two methods of login authentication; SQL Server Authentication and Windows Authentication. The configuration where both SQL Server Authentication logins and Windows Authentication logins are accepted is known as Mixed Mode Authentication. SQL Server PDW only supports Mixed Mode Authentication, which means that neither SQL Server Authentication nor Windows Authentication can be disabled or turned off. However, you do not have to permit any SQL Server logins except the administrative sa login and the logins used internally by the SQL Server PDW components. And you do not have to create any Windows logins.

This topic briefly defines SQL Server authentication, but primarily describes the configuration necessary to support Windows Authentication.

SQL Server Authentication

SQL Server Authentication is an authentication method which does not rely upon Windows credentials. Logins are created in SQL Server PDW, and SQL Server PDW stores the login name, and a hash of the login password. When connecting using a SQL Server Authentication login, the client submits the login name and password to SQL Server PDW in an encrypted string, which PDW uses to authenticate the identity of the user. Separate SQL Server logins should be created for each person connecting to SQL Server PDW. Windows Authentication is preferred over SQL Server Authentication for many reasons. because SQL Server Authentication requires users to remember a second identity and password for PDW, and enter credentials at the time of connection, and because SQL Server Authentication does not support all of the security features, such as Kerberos.

  • SQL Server Authentication requires users to remember a second identity and password for PDW, and enter credentials at the time of connection.

  • SQL Server Authentication does not support all of the security features, such as Kerberos.

  • SQL Server Authentication requires transmitting the login password over the network when connecting. The password is encrypted, but is an point of potential attack.

  • By using Windows Authentication, Windows groups can be created at the domain level, and a login can be created on SQL Server for the entire group. Managing access from at the domain level can simplify account administration.

For additional information about authentication modes, see Choose an Authentication Mode.

The following tools do not yet support Windows Authentication and must be used with SQL Server Authentication logins.

  • Informatica

  • BOBJ connectors

  • The internal PDW Appliance Verification (PAV) tool

Windows Authentication

Windows Authentication (sometimes known as Integrated Authentication) lets you create logins that represent Windows users, or Windows groups. The authentication of the connecting Windows user is performed by the client operating system and domain when the client logs into the client computer. The connecting Windows users pass Windows tokens to SQL Server PDW which include both the authenticated Windows user identity and the security identifier (SID) for the Windows groups that the Windows user belongs to. SQL Server PDW does not have to verify the identity of the connecting user but does connect to the authenticating Active Directory to validate the tokens. SQL Server PDW accepts the identity from any Windows domain that is trusted by the SQL Server PDW appliance domain. SQL Server PDW must still confirm that the connecting user has a login, either for the Windows user account, or for a Windows group. If fully configured, Windows Authentication supports Kerberos authentication. If Kerberos authentication fails, the connection automatically attempts NTLM authentication. Kerberos authentication is preferred.

Analytics Platform System contains a single appliance domain and this domain participates in user authentication. The appliance domain contains organizational units (OUs) for each region within the appliance. The PDW OU contains the virtual machines that are the logical level nodes of the PDW appliance: control (CTL) and compute nodes (CMP). For more information on the PDW appliance architecture, see Understanding SQL Server PDW (SQL Server PDW). The appliance active directory must be configured, as well as the corporate domain in order for customers to be able to use Windows Authentication with PDW.

To connect to PDW with their domain accounts using Windows Authentication, the PDW appliance domain must trust the corporate domain where Windows users are being authenticated. A one-way forest trust is the minimal trust required for Kerberos to work with Windows Authentication. Users from other domains that are in a transitive trust with that corporate domain will also be able to connect to the PDW appliance. There are other configurations that can enable Integrated Authentication, such as 2-way forest or external trust for Kerberos, 1-way external trust, or account mirroring for NTLM that are not discussed in this topic.

For Kerberos to work, an appropriate Service Principal Name (SPN) must be registered on the APS appliance domain controller. In PDW we provide this automatically. During setup PDW registers a default SPN in the appliance Active Directory Domain Services based on the internal name of the CTL node and port 17001. Customers can use this SPN by creating a forward lookup zone (FLZ) in their corporate DNS that uses internal PDW names, and also adding the appropriate _ldap and _kerberos SRV records. To use some other, more meaningful names for accessing PDW nodes, aliases can be created. For Kerberos to work, PDW must be accessed by the DNS name that corresponds to a registered SPN, or by an alias that points to such a name. If an IP address is used, fallback to NTLM authentication may occur.

When a Windows authentication connection is initiated, the system will first try to use Kerberos as a more secure protocol. If Kerberos cannot be used, authentication will fall back to NTLM. If NTLM cannot be used either (e.g. because there is no trust established between the domains), then Windows authentication will not be possible. The principal reasons Kerberos could fail, are:

  • Because a 1-way external domain trust was established between domains rather than a forest trust.

  • Because the required SRV records were not properly added to the corporate DNS.

  • Because an IP address was used to connect rather than the DNS name that corresponds to registered SPN.

For step-by-step instructions to configure domains for Windows Authentication, see Security - Configure Domain Trusts (SQL Server PDW).


Security - Configure Domain Trusts (SQL Server PDW)

[image]

Security - Configure Domain Trusts (SQL Server PDW)
See Also Send Feedback

Perform the following steps to configure SQL Server PDW to accept Windows Authentication logins from a corporate domain, and configure a corporate DNS server to provide authentication to SQL Server PDW. For all steps, login as a domain administrator or other high privileged user. This process includes the following steps:

  • Configure a Forwarder on PDW DNS

  • Configure a Forward Lookup Zone on the Corporate DNS

  • Configure a Reverse Lookup Zone on the Corporate DNS

  • Add Records for CTL and AD Nodes to the New FLZ on the Corporate DNS Server

  • Add SRV Records Needed for Kerberos on the Corporate DNS Server

  • Add Aliases for the CTL and AD Nodes to the Corporate DNS

  • Create a Trust

  • Validate the Trust

Steps on the SQL Server PDW

Configure a Forwarder on PDW DNS

  1. Log in to appliance_domain-HST01 as the appliance domain administrator.

  2. Open the Hyper-V Manager program (virtmgmt.msc).

    1. Connect to appliance_domain-HST01, if not connected by default.

    2. In the Virtual Machines pane, right-click appliance_domain-AD01, and click Connect.

    3. Log on to the appliance_domain-AD01 as the appliance domain administrator.

  3. While on appliance_domain-AD01, open the DNS Manager (dnsmgmt.msc).

  4. Right-click your DNS Server (appliance_domain-AD01), and then click Properties.

  5. On the Advanced tab, clear the checkbox for Disable recursion (also disables forwarders) to permit forwarders, and then click Apply.

  6. On the Forwarders tab, click Edit.

  7. In the Edit Forwarders dialog box, type the IP address of the DNS server for the corporate domain that will provide authentication.

  8. Click OK to close the Edit Forwarders dialog box, and OK to close the DNS Properties dialog box.

  9. While on appliance_domain-HST01, go back to the Hyper-V Manager.

    1. Connect to appliance_domain-HST02, if not connected by default.

    2. In the Virtual Machines pane, right-click appliance_domain-AD02, and then click Connect.

    3. Log on to the appliance_domain-AD02 as the appliance domain administrator.

    4. While on appliance_domain-AD02, perform steps 3 – 8 above.

PDW Windows Authentication

Steps to Set Up Active Directory and DNS on the Corporate Domain

Configure a Forward Lookup Zone (FLZ) on the Corporate DNS

  1. Open the DNS Manager (dnsmgmt.msc) on a corporate DC of the domain that will provide authentication.

  2. In the left pane, expand the DNS Server for the authenticating domain, right-click Forward Lookup Zones, and then click New Zone.

  3. Complete the New Zone Wizard to create a Primary zone using the name of the appliance domain as the Zone Name. The name will be in the format: F12345.pdw.local where F12345 is the appliance domain. See your domain administrator for the preferred selection for the other options.

Tip Tip

At this point you should be able to successfully use ping to connect to the IP addresses of the SQL Server PDW management and control nodes, in the format ping xxx.xxx.xxx.xxx

Configure a Reverse Lookup Zone (RLZ) on the Corporate DNS

  1. Continue using the DNS Manager on a corporate DC of the domain that will provide authentication. In the left pane, expand the DNS Server for the authenticating domain, right-click Reverse Lookup Zones, and then click New Zone.

  2. Complete the New Zone Wizard to create a Primary zone, and specifying an IPv4 Reverse Lookup Zone. In the Network ID box, provide enough of the IP address to indicate the IP addresses that belong to this zone. (Typically this will be the first 3 octets of the SQL Server PDW IP addresses.) See your domain administrator for the preferred selection for the other options or if your IP addresses are not typical.

PDW Windows Authentication RLZ

Add Records for CTL and Appliance AD Nodes to the New FLZ on the Corporate DNS Server

  1. Continue using the DNS Manager on a corporate DC of the domain that will provide authentication. In the left-pane, expand Forward Lookup Zones, and then right-click on the new FLZ of SQL Server PDW (that you created in a previous step), and then click New Host (A or AAAA).

    1. In the New Host dialog box, in the Name box, type the name of the PDW control node, in the format PDW_region-ctl01, such as P12345-ctl01.

    2. In the IP address box, type the external IP address of the control node.

    3. Check the Create associated pointer (PTR) record box.

    4. Click Add Host to create a reverse lookup entry for the control node.

  2. Repeat the New Host process to create a pointer record for both appliance_domain-AD01 and appliance_domain-AD02.

PDW Windows Authentication Host
Tip Tip

At this point you should be able to successfully use ping to connect to the FQDN's of the SQL Server PDW Control node, appliance_domain-AD01, and appliance_domain-AD02, by using the following ping commands:

  • ping PDW_region-CTL01.appliance_domain.local

  • ping appliance_domain-AD01.appliance_domain.local

  • ping appliance_domain-AD02.appliance_domain.local

Add SRV Records Needed for Kerberos on the Corporate DNS Server

  1. Continue using the DNS Manager on a corporate DC of the domain that will provide authentication. Right-click on the FLZ for PDW created earlier, and then click New Domain.

  2. In the New DNS Domain dialog box, type dc._msdcs, and then click OK.

  3. In the left-pane, expand Forward Lookup Zones, expand your PDW zone name, expand _msdcs node, right click dc, and click Other New Records.

  4. In the Resource Record Type dialog box, click Service Location (SRV), and then click Create Record.

  5. In the Service box, select _ldap, and in the Host offering this service box, enter the full DNS name of the appliance_domain-AD01 node, suffixed by a dot. For example, F12345-AD01.F12345.pdw.local. Click OK.

  6. In the Resource Record Type dialog box, click Service Location (SRV), and then click Create Record.

  7. In the Service box, select _kerberos, and in the Host offering this service box, enter the full DNS name of the appliance_domain-AD01 node, suffixed by a dot. For example, F12345-AD01.F12345.pdw.local. Click OK.

  8. To close the Resource Record Type dialog box, click Done.

  9. Repeat steps 4 – 8 to create _ldap and _kerberos records for appliance_domain-AD02.

PDW Windows Authentication Kerberos

Add Aliases for the CTL and AD Nodes to the Corporate DNS

  1. Continue using the DNS Manager on a corporate DC of the domain that will provide authentication. Right-click on the a FLZ of the authenticating corporate domain, and then click New Alias (CNAME).

  2. In the New Resource Record dialog box, in the Alias name box, type the name that you wish to use for the control node. This example uses PdwControl.

  3. In the Fully qualified domain name (FQDN) for target host box, type the full internal name of the control node, or click Browse to find it on the DNS (in the FLZ with the SQL Server PDW appliance domain name).

  4. In the New Resource Record dialog box, in the Alias name box, type the name that you wish to use for the appliance_domain-AD01. This example uses FabAd01.

  5. In the Fully qualified domain name (FQDN) for target host box, type the full internal name of the appliance_domain-AD01, or click Browse to find it on the DNS (in the FLZ with the SQL Server PDW appliance domain name).

  6. Repeat steps 4 – 6 for the appliance_domain-AD02 to create a FabAd02 alias.

    Note Note

    These aliases should be used when accessing the PDW from corporate domain.

PDW Windows Authentication Alias
Tip Tip

At this point you should be able to successfully use ping to connect to the alias names of the SQL Server PDW Control and Active Directory nodes, in the format ping PdwControl and ping FabAd01 and ping FabAD02

Create a Trust

  1. On a corporate domain controller, open Active Directory Domains and Trusts (domain.msc).

  2. In the left pane, right-click on the domain name, and then click Properties.

  3. On the Trusts tab, click New Trust, and then complete the New Trust Wizard providing appliance domain administrator credentials when asked.

    1. For Trust Name enter the fully qualified appliance domain name, in the format F1234.pdw.local

    2. Choose a Forest trust.

    3. Choose One-way: incoming.

      Note Note

      A Two-way trust can be created but it is not necessary.

    4. Choose Both this domain and the specified domain. You will be prompted for administrator credentials on the appliance domain.

    5. In most cases choose Forest-wide authentication. Choose Selective authentication if you wish to add additional restrictions on authentication.

Validate the Trust

  1. Select the new trust in the list of incoming trusts, and then click Properties. Click Validate.

  2. Choose Yes, validate the incoming trust and enter appliance domain administrator credentials.

PDW Windows Authentication AD Services

See Also

Concepts

Security - Windows Authentication and SQL Server Authentication (SQL Server PDW)


Transparent Data Encryption (SQL Server PDW)

[image]

Transparent Data Encryption (SQL Server PDW)
See Also Send Feedback

You can take several precautions to help secure the database such as designing a secure system, encrypting confidential assets, and building a firewall around the database servers. However, in a scenario where the physical media (such as drives or backup tapes) are stolen, a malicious party can just restore or attach the database and browse the data. One solution is to encrypt the sensitive data in the database and protect the keys that are used to encrypt the data with a certificate. This prevents anyone without the keys from using the data, but this kind of protection must be planned in advance.

Transparent data encryption (TDE) performs real-time I/O encryption and decryption of the data and transaction log files and the special PDW log files. The encryption uses a database encryption key (DEK), which is stored in the database boot record for availability during recovery. The DEK is a symmetric key secured by using a certificate stored in the master database of the SQL Server PDW. TDE protects data "at rest", meaning the data and log files. It provides the ability to comply with many laws, regulations, and guidelines established in various industries. This enables software developers to encrypt data by using AES and 3DES encryption algorithms without changing existing applications.

Important note Important

TDE does not provide encryption for data traveling between the client and the SQL Server PDW. For more information about how to encrypt data between the client and SQL Server PDW, see PDW Certificate Provisioning (Analytics Platform System).

TDE does not encrypt data while it is moving or in use. Internal traffic between PDW components inside the SQL Server PDW is not encrypted. Data temporarily stored in memory buffers is not encrypted. To mitigate this risk, control physical access and connections to the SQL Server PDW.

After it is secured, the database can be restored by using the correct certificate.

Note Note

When you create a certificate for TDE, you should immediately back it up, together with the associated private key. If the certificate ever becomes unavailable or if you must restore or attach the database on another server, you must have backups of both the certificate and the private key or you will not be able to open the database. The encrypting certificate should be retained even if TDE is no longer enabled on the database. Even though the database is not encrypted, parts of the transaction log may still remain protected, and the certificate may be needed for some operations until the full backup of the database is performed. A certificate that has exceeded its expiration date can still be used to encrypt and decrypt data with TDE.

Encryption of the database file is performed at the page level. The pages in an encrypted database are encrypted before they are written to disk and decrypted when read into memory. TDE does not increase the size of the encrypted database.

The following illustration shows the hierarchy of keys for TDE encryption:

Displays the hierarchy described in the topic.

Using Transparent Data Encryption

To use TDE, follow these steps. The first three steps are only done once, when preparing SQL Server PDW to support TDE.

  1. Create a master key in the master database.

  2. Use sp_pdw_database_encryption to enable TDE on the SQL Server PDW. This operation modifies the temporary databases in order to ensure the protection of future temporary data, and will fail if attempted when there are any active sessions that have temporary tables. sp_pdw_database_encryption turns on user data masking in PDW system logs. (For more information about user data masking in PDW system logs, see sp_pdw_log_user_data_masking (SQL Server PDW).)

  3. Use sp_pdw_add_network_credentials to create a credential that can authenticate and write to the share where the backup of the certificate will be stored. If a credential already exist for the intended storage server, the existing credential can be used.

  4. In the master database, create a certificate protected by the master key.

  5. Backup the certificate to the storage share.

  6. In the user database, create a database encryption key and protect it by the certificate that is stored in the master database.

  7. Use the ALTER DATABASE statement to encrypt the database using TDE.

The following example illustrates encrypting the AdventureWorksPDW2012 database using a certificate named MyServerCert, created in SQL Server PDW.

First: Enable TDE on the SQL Server PDW. This is only necessary once.

Copy image Copy Code
USE master;
GO

-- Create a database master key in the master database
CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<UseStrongPasswordHere>';
GO

-- Enable encryption for PDW
EXEC sp_pdw_database_encryption 1;
GO

-- Add a credential that can write to the share
-- A credential created for a backup can be used if you wish
EXEC sp_pdw_add_network_credentials 'SECURE_SERVER', '<domain>\<Windows_user>', '<password>';

Second: Create and backup a certificate in the master database. This is only required once. You can have a separate certificate for each database (recommended), or you can protect multiple databases with one certificate.

Copy image Copy Code
-- Create certificate in master
CREATE CERTIFICATE MyServerCert WITH SUBJECT = 'My DEK Certificate';
GO

-- Back up the certificate with private key
BACKUP CERTIFICATE MyServerCert
    TO FILE = '\\SECURE_SERVER\cert\MyServerCert.cer'
    WITH PRIVATE KEY
    (
        FILE = '\\SECURE_SERVER\cert\MyServerCert.key',
        ENCRYPTION BY PASSWORD = '<UseStrongPasswordHere>'
    )
GO

Last: Create the DEK and use ALTER DATABASE to encrypt a user database. This is repeated for each database that is protected by TDE.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO

CREATE DATABASE ENCRYPTION KEY
WITH ALGORITHM = AES_128
ENCRYPTION BY SERVER CERTIFICATE MyServerCert;
GO

ALTER DATABASE AdventureWorksPDW2012 SET ENCRYPTION ON;
GO

The encryption and decryption operations are scheduled on background threads by SQL Server. You can view the status of these operations using the catalog views and dynamic management views in the list that appears later in this topic.

Caution note Caution

Backup files of databases that have TDE enabled are also encrypted by using the database encryption key. As a result, when you restore these backups, the certificate protecting the database encryption key must be available. This means that in addition to backing up the database, you have to make sure that you maintain backups of the server certificates to prevent data loss. Data loss will result if the certificate is no longer available.

Commands and Functions

The TDE certificates must be encrypted by the database master key to be accepted by the following statements.

The following table provides links and explanations of TDE commands and functions.

Command or function | Purpose
CREATE DATABASE ENCRYPTION KEY (SQL Server PDW) | Creates a key that is used to encrypt a database.
ALTER DATABASE ENCRYPTION KEY (SQL Server PDW) | Changes the key that is used to encrypt a database.
DROP DATABASE ENCRYPTION KEY (SQL Server PDW) | Removes the key that was used to encrypt a database.
ALTER DATABASE (SQL Server PDW) | Explains the ALTER DATABASE option that is used to enable TDE.

Catalog Views and Dynamic Management Views

The following table shows TDE catalog views and dynamic management views.

Catalog view or dynamic management view | Purpose
sys.databases (SQL Server PDW) | Catalog view that displays database information.
sys.certificates (SQL Server PDW) | Catalog view that shows the certificates in a database.
sys.dm_pdw_nodes_database_encryption_keys (SQL Server PDW) | Dynamic management view that provides information for each node, about the encryption keys used in a database, and the state of encryption of a database.

Permissions

Each TDE feature and command has individual permission requirements, described in the tables shown earlier.

Viewing the metadata involved with TDE requires the CONTROL SERVER permission.

Considerations

While a re-encryption scan for a database encryption operation is in progress, maintenance operations to the database are disabled.

You can find the state of the database encryption using the sys.dm_pdw_nodes_database_encryption_keys dynamic management view. For more information, see the Catalog Views and Dynamic Management Views section earlier in this topic).

Restrictions

The following operations are not allowed during the CREATE DATABASE ENCRYPTION KEY, ALTER DATABASE ENCRYPTION KEY, DROP DATABASE ENCRYPTION KEY, or ALTER DATABASE...SET ENCRYPTION statements.

  • Dropping the database.

  • Using an ALTER DATABASE command.

  • Starting a database backup.

  • Starting a database restore.

The following operations or conditions will prevent the CREATE DATABASE ENCRYPTION KEY, ALTER DATABASE ENCRYPTION KEY, DROP DATABASE ENCRYPTION KEY, or ALTER DATABASE...SET ENCRYPTION statements.

  • An ALTER DATABASE command is executing.

  • Any data backup is running.

When creating database files, instant file initialization is not available when TDE is enabled.

Areas Not Protected by TDE

TDE does not protect external tables.

TDE does not protect diagnostic sessions. Users should be careful not to queries with sensitive parameters while diagnostic sessions are in use. Diagnostic sessions that reveal sensitive information should be dropped as soon as they are no longer needed.

Data protected by TDE is decrypted when placed in SQL Server PDW memory. Memory dumps are created when certain problems occur on appliance. Dump files represent the content of the memory at the time of the problem appearance and can contain sensitive data in an unencrypted form. The content of memory dumps should be reviewed before they are shared with others.

The master database is not protected by TDE. Though the master database does not contain user data, it does contain information such as login names.

Transparent Data Encryption and Transaction Logs

Enabling a database to use TDE has the effect of zeroing out the remaining part of the virtual transaction log to force the next virtual transaction log. This guarantees that no clear text is left in the transaction logs after the database is set for encryption. You can find the status of the log file encryption on each PDW node by viewing the encryption_state column in the sys.dm_pdw_nodes_database_encryption_keys view, as in this example:

Copy image Copy Code
WITH dek_encryption_state AS
(
    SELECT ISNULL(db_map.database_id, dek.database_id) AS database_id, encryption_state
    FROM sys.dm_pdw_nodes_database_encryption_keys AS dek
        INNER JOIN sys.pdw_nodes_pdw_physical_databases AS node_db_map
           ON dek.database_id = node_db_map.database_id AND dek.pdw_node_id = node_db_map.pdw_node_id
        LEFT JOIN sys.pdw_database_mappings AS db_map
            ON node_db_map .physical_name = db_map.physical_name
        INNER JOIN sys.dm_pdw_nodes AS nodes
            ON nodes.pdw_node_id = dek.pdw_node_id
    WHERE dek.encryptor_thumbprint <> 0x
)
SELECT TOP 1 encryption_state
       FROM dek_encryption_state
       WHERE dek_encryption_state.database_id = DB_ID('AdventureWorksPDW2012 ')
       ORDER BY (CASE encryption_state WHEN 3 THEN -1 ELSE encryption_state END) DESC;

All data written to the transaction log before a change in the database encryption key will be encrypted by using the previous database encryption key.

PDW Activity Logs

SQL Server PDW maintains a set of logs intended for troubleshooting. (Note, this is not the transaction log, the SQL Server error log or the Windows event log.) These PDW activity logs can contain full statements in clear text, some of which can contain user data. Typical examples are INSERT and UPDATE statements. Masking of user data can be explicitly turned on or off by using sp_pdw_log_user_data_masking. Enabling encryption on SQL Server PDW automatically turns on the masking of user data in PDW activity logs in order to protect them. sp_pdw_log_user_data_masking can also be used to mask statements when not using TDE, but that is not recommended because it significantly reduces the ability of the Microsoft Support Team to analyze problems.

Transparent Data Encryption and the tempdb System Database

The tempdb system database is encrypted when encryption is enabled by using sp_pdw_database_encryption. This is required before any database can use TDE. This might have a performance effect for unencrypted databases on the same instance of SQL Server PDW.

Key Management

The database encryption key (DEK) is protected by the certificates stored in the master database. These certificates are protected by the database master key (DMK) of the master database. The DMK needs to be protected by the service master key (SMK) in order to be used for TDE.

The system can access the keys without requiring human intervention (such as providing a password). If the certificate is not available, the system will output an error explaining that the DEK cannot be decrypted until the proper certificate is available.

When moving a database from one appliance to another, the certificate used to protect its’ DEK must be restored first on the destination server. Then the database can be restored as usual. For more information, see the standard SQL Server documentation, at Move a TDE Protected Database to Another SQL Server.

Certificates used to encrypt DEKs should be retained as long as there are database backups that use them. Certificate backups must include the certificate private key, because without the private key a certificate cannot be used for database restore. Those certificate private key backups are stored in a separate file, protected by a password that must be provided for certificate restore.

Restoring the master Database

The master database can be restored using DWConfig, as part of the disaster recovery.

  • If the control node hasn’t changed, that is if the master database is restored on the same and unchanged appliance from which the backup of master database was taken, the DMK and all the certificates will be readable without additional action.

  • If the master database is restored on a different appliance, or if the control node has been changed since the backup of the master database, additional steps will be required in order to regenerate the DMK.

    1. Open the DMK:

      Copy image Copy Code
      OPEN MASTER KEY DECRYPTION BY PASSWORD = '<password>';

    2. Add the encryption by SMK:

      Copy image Copy Code
      ALTER MASTER KEY
    ADD ENCRYPTION BY SERVICE MASTER KEY;

    3. Restart the appliance.

Upgrade and Replacing Virtual Machines

If a DMK exists on the appliance on which Upgrade or Replace VM was performed, DMK password must be provided as a parameter.

Example of the upgrade action. Replace ********** with your DMK password.

Copy image Copy Code
setup.exe /Action=ProvisionUpgrade … DMKPassword='**********'

Example of the replace virtual machine action.

Copy image Copy Code
setup.exe /Action=ReplaceVM … DMKPassword='**********'

During upgrade, if a user DB is encrypted and the DMK password is not provided, the upgrade action will fail. During replace, if the correct password is not provided when a DMK exists, the operation will skip the DMK recovery step. All the other steps will be completed at the end of the replace VM action, however the action will report failure at the end to indicate that additional steps are required. In the setup logs (located in \ProgramData\Microsoft\Microsoft SQL Server Parallel Data Warehouse\100\Logs\Setup\<time-stamp>\Detail-Setup), the following warning will be shown near the end.

*** WARNING *** DMK is detected in master database, but could not be recovered automatically! The DMK password was either not provided or is incorrect!

Please execute manually these statements in PDW and restart appliance after that in order to recover DMK:

OPEN MASTER KEY DECRYPTION BY PASSWORD = '<DMK password>';

ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY;

Use the steps in the Restoring the master Database paragraph to recover the database, and then restart the appliance.

If the DMK existed before but was not recovered after the action, the following error message will be raised when a database is queried.

Msg 110806;

A distributed query failed: Database '<db_name>' cannot be opened due to inaccessible files or insufficient memory or disk space. See the SQL Server errorlog for details.

Performance Impact

The performance impact of TDE varies with the type of data you have, how it is stored, and the type of workload activity on the SQL Server PDW. When protected by TDE, the I/O of reading and then decrypting data or the encrypting and then writing data is a CPU intensive activity and will have more impact when other CPU intensive activities are happening at the same time. Because TDE encrypts tempdb, TDE can affect the performance of databases that are not encrypted. To get an accurate idea of performance, you should test the entire system with your data and query activity.

Related Content

The following links contain general information about how SQL Server manages encryption. These topics can help you understand SQL Server encryption, but these topics do not have information specific to SQL Server PDW and they discuss features that are not present in SQL Server PDW.

  • SQL Server Encryption

  • Encryption Hierarchy

  • SQL Server and Database Encryption Keys (Database Engine)

  • The Data Compression and Transparent Data Encryption section, of Data Compression: Strategy, Capacity Planning and Best Practices

See Also

Reference

ALTER DATABASE (SQL Server PDW)
CREATE MASTER KEY (SQL Server PDW)
CREATE DATABASE ENCRYPTION KEY (SQL Server PDW)
BACKUP CERTIFICATE (SQL Server PDW)
sp_pdw_database_encryption (SQL Server PDW)
sp_pdw_database_encryption_regenerate_system_keys (SQL Server PDW)
sp_pdw_log_user_data_masking (SQL Server PDW)
sys.certificates (SQL Server PDW)
sys.dm_pdw_nodes_database_encryption_keys (SQL Server PDW)


Remote Table Copy (SQL Server PDW)

[image]

Remote Table Copy (SQL Server PDW)
See Also Send Feedback

Describes how to use the remote table copy feature to copy tables from SQL Server PDW databases to remote (non-appliance) SMP SQL Server databases. Use remote table copy to enables hub and spoke scenarios for SQL Server PDW.

Contents

  • Understand Remote Table Copy

  • Requirements for Using Remote Table Copy

  • Configure Remote Table Copy

  • Perform a Remote Table Copy

Understand Remote Table Copy for SQL Server PDW

Remote table copy is a feature of SQL Server PDW that enables Hub and Spoke scenarios by copying the results of a SQL SELECT statement to a table in an SMP database. The remote table copy is initiated with the CREATE REMOTE TABLE AS SELECT (SQL Server PDW) statement.

[TOP]

Requirements for Using Remote Table Copy

You can use remote table copy to copy tables from SQL Server PDW to a SQL Server database when these conditions are met:

  • The destination database must be an instance of Microsoft® SQL Server® that is running on a Microsoft Windows® system that can connect to the SQL Server PDW appliance but does not reside on a server within the appliance. The remote SQL Server can be connected to the SQL Server PDW using the InfiniBand network or through the Ethernet network.

  • The data to be copied must be selectable using a single valid SQL Server PDW SELECT (SQL Server PDW) statement.

  • The destination server must be a non-appliance server. Data cannot be copied directly from one appliance to another using the instructions in this topic.

  • The destination server must be accessible to all nodes on the appliance's Infiniband network.

[TOP]

Configure Remote Table Copy

To use remote table copy, you need to purchase and configure a Windows server, configure SQL Server on the Windows server, and configure SQL Server PDW. Use the following links to perform these three configuration steps.

  1. Configure an External Windows System To Receive Remote Table Copies Using InfiniBand (SQL Server PDW)

  2. Configure an External SMP SQL Server to Receive Remote Table Copies (SQL Server PDW)

  3. Configure SQL Server PDW for Remote Table Copies (SQL Server PDW)

Perform a Remote Table COpy

To perform a remote table copy, use the CREATE REMOTE TABLE AS SELECT (SQL Server PDW) SQL statement. Examples are included with the CREATE REMOTE TABLE topic.

See Also


Configure an External Windows System To Receive Remote Table Copies Using InfiniBand (SQL Server PDW)

[image]

Configure an External Windows System To Receive Remote Table Copies Using InfiniBand (SQL Server PDW)
See Also Send Feedback

Describes how to purchase and configure a non-appliance Windows system connected using the InfiniBand network for use with the remote table copy feature in SQL Server PDW. The Windows system will host the SQL Server database that receives the remote table copy from a SQL Server PDW database. It is purchased separately from the appliance and connected to the appliance InfiniBand network.

Note Note

Connecting through the InfiniBand network is not required for using remote table copy. Connecting through the Ethernet network can be done if the Ethernet bandwidth meets your needs.

This topic describes one of the configuration steps for configuring remote table copy. For a list of all the configuration steps, see Remote Table Copy (SQL Server PDW)

Before You Begin

Before you configure the external Windows system, you must:

  1. Purchase or provide a Windows system that will receive the remote copies.

  2. Rack the Windows system in the Control rack (if there is enough space) or close enough to the appliance so that you can connect it to the appliance InfiniBand network.

  3. Purchase InfiniBand cables and an InfiniBand network adapter from your appliance hardware vendor. We recommend purchasing a network adapter with two ports for fault tolerance when receiving the exported data. A two port network adapter is recommended, but is not a requirement.

Configure an External Windows System To Receive Remote Table Copies

To configure the external Windows system, use the following steps:

  1. Install the InfiniBand network adapter into your Windows system.

  2. Connect the InfiniBand network adapter to the main InfiniBand switch in the Control rack using InfiniBand cables.

  3. Install and configure the appropriate Windows driver for the InfiniBand network adapter.

    InfiniBand drivers for Windows are developed by the OpenFabrics Alliance, an industry consortium of InfiniBand vendors. The correct driver may have been distributed with your InfiniBand adapter. If not, the driver can be downloaded from www.openfabrics.org.

  4. Configure the IP address for each port on the adapter. SMP systems are required to use static IP addresses from a range of addresses reserved for this purpose. Configure the first port according to the following parameters:

    • IP network address: 172.16.132.x

    • IP subnet mask: 255.255.128.0

    • IP host range: 1-254

    For InfiniBand network adapters with two ports, configure the second port according to the following parameters:

    • IP network address: 172.16.132.x

    • IP subnet mask: 255.255.128.0

    • IP host range: 1-254

  5. If a two port adapter is used, or multiple external Windows systems are connected to an appliance, assign each system a different host number within each IP subnet.

[TOP]

See Also


Configure an External SMP SQL Server to Receive Remote Table Copies (SQL Server PDW)

[image]

Configure an External SMP SQL Server to Receive Remote Table Copies (SQL Server PDW)
See Also Send Feedback

Describes how to configure an external SQL Server instance to receive remote table copies from SQL Server PDW.

This topic describes one of the configuration steps for configuring remote table copy. For a list of all the configuration steps, see Remote Table Copy (SQL Server PDW)

Before You Begin

Before you can configure the external SQL Server, you must:

  • Have a Windows system with SQL Server 2008 Enterprise Edition or a later version ready to be installed or already installed. The Windows system must already be configured according to the instructions in Configure an External Windows System To Receive Remote Table Copies Using InfiniBand (SQL Server PDW)

  • A Windows admin account with the ability to configure the SQL Server instance and the Windows system.

  • A SQL Server login account (if SQL Server is already installed) with the ability to create logins and grant permissions on the destination database(s).

Configure an External SMP SQL Server To Receive Remote Table Copies

The remote table copy feature copies tables from the SQL Server PDW appliance to an external SMP SQL Server database that is running on a Windows system. After configuring the external Windows system to receive remote table copies, the next step is to install and configure SQL Server onto the Windows system.

To configure SQL Server, use the following steps:

  1. Install SQL Server 2008 Enterprise Edition or a later version on the Windows system. This is referred to as the SMP SQL Server.

  2. Configure SQL Server to accept TCP/IP connections on a fixed TCP port. This configuration is disabled by default and must be enabled to allow SQL Server PDW to connect to the SMP SQL Server.

  3. Either disable Windows firewall or configure the SMP SQL Server TCP port so that it will work with Windows firewall enabled.

  4. Configure SQL Server to allow SQL Server authentication mode. The parallel data export always uses SQL Server accounts for authentication.

  5. Determine a SQL Server account on the SMP SQL Server that will be used for authentication. Grant that account the privilege to create, drop, and insert data into tables in the destination database for the parallel data export operation.

[TOP]

Best Practices for SMP SQL Server Configuration for Remote Table Copy

When configuring the SMP SQL Server to receive remote table copies, use the following best practices to improve performance.

  1. Follow best practices as documented in SQL Server product documentation. For example, enable data encryption. For more information about securing SQL Server, see Securing SQL Server on MSDN.

  2. Use the bulk-logged or simple recovery model.

    During the parallel data export operations, data is bulk inserted into the newly created destination table. For maximum performance during the bulk insert, set the destination database to use the bulk-logged or the simple recovery model.

  3. Use the batch_size option to reclaim log space.

    Although the bulk-logged or simple recovery models use minimal logging for the bulk inserted data, some logging still occurs. To prevent the log files from growing too large, use the SQL Server batch_size option to periodically reclaim log space.

[TOP]

See Also


Configure SQL Server PDW for Remote Table Copies (SQL Server PDW)

[image]

Configure SQL Server PDW for Remote Table Copies (SQL Server PDW)
See Also Send Feedback

Describes how to configure SQL Server PDW to use the remote table copy feature to copy tables to SMP SQL Server databases on non-appliance servers.

This topic describes one of the configuration steps for configuring remote table copy. For a list of all the configuration steps, see Remote Table Copy (SQL Server PDW)

Before You Begin

In order to configure SQL Server PDW to use remote table copy, you must:

  • Have a Analytics Platform System administrator account with the ability to log directly into the appliance_domain-AD01 and appliance_domain-AD02 nodes.

  • Know the host name or IP name of the destination server.

Configure SQL Server PDW for Remote Table Copy: Update Host Names in DNS

The CREATE REMOTE TABLE statement, used for remote table copies, specifies the destination server by using either the IP address or the IP name of the SMP Windows system. To use the IP name, you need to add entries for successful name resolution to the DNS server.

The following steps outline how to update the DNS server.

  1. Log on to the active AD node (normally appliance_domain-AD01).

  2. Open the DNS Manager. This is located under Administrative Tools in the Start menu.

  3. Use the DNS Manager to add the IP name.

See Also

Tasks

Security - Configure Domain Trusts (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Use a DNS Forwarder to Resolve Non-Appliance DNS Names (Analytics Platform System)


System Databases (SQL Server PDW)

[image]

System Databases (SQL Server PDW)
See Also Send Feedback

System databases are databases that SQL Server PDW uses to manage users, databases, and temporary data. SQL Server PDW includes the following system databases: master, and tempdb.

System Database Topics

  • master Database (SQL Server PDW)

  • tempdb Database (SQL Server PDW)

See Also


Reserved Database Names (SQL Server PDW)

[image]

Reserved Database Names (SQL Server PDW)
See Also Send Feedback

A reserved database name is a name that cannot be assigned to a user database in SQL Server PDW.

List of reserved database names:

  • master

  • model

  • msdb

  • tempdb

  • pdwtempdb

  • pdwtempdb1

  • pdwtempdb2

  • DWConfiguration

  • DWDiagnostics

  • DWQueue

See Also

Concepts

Database Administration (SQL Server PDW)
System Databases (SQL Server PDW)


tempdb Database (SQL Server PDW)

[image]

tempdb Database (SQL Server PDW)
See Also Send Feedback

tempdb is a SQL Server PDW system database that stores local temporary tables for user databases. Temporary tables are often used to improve query performance. For example, you can use a temporary table to modularize a script, and reuse computed data.

For more information about system databases, see System Databases (SQL Server PDW)

Contents

  • Basics

  • Limitations and Restrictions

  • Related Tasks

Key Terms and Concepts

local temporary table

A local temporary table uses the # prefix before the table name and is a temporary table created by a local user session. Each session can only access the data in local temporary tables for its own session.

Each session can view the metadata for local temporary tables in all sessions. For example, all sessions can view the metadata for all local temporary tables with the SELECT * FROM tempdb.sys.tables query.

global temporary table

Global temporary tables, supported in SQL Server with the ## syntax, are not supported in this release of SQL Server PDW.

pdwtempdb

pdwtempdb is the database that stores local temporary tables.

PDW does not implement temporary tables by using the SQL Server tempdb database. Instead, PDW stores them in a database called pdwtempdb. This database exists on each Compute node and is invisible to the user through the PDW interfaces. In the Admin Console, on the storage page, you will see these accounted for in a PDW system database called tempdb-sql.

tempdb

tempdb is the SQL Server tempdb database. It uses minimal logging. SQL Server uses tempdb on the Compute nodes to store temporary tables that it needs in the course of performing SQL Server operations.

SQL Server PDW drops tables from tempdb when:

  • The DROP TABLE statement is executed.

  • A session is disconnected. Only temporary tables for the session are dropped.

  • The appliance is shutdown.

  • The Control node has a cluster failover.

TOP

General Remarks

SQL Server PDW performs the same operations on temporary tables and permanent tables unless explicitly stated otherwise. For example, the data in local temporary tables, just like permanent tables, is either distributed or replicated across the Compute nodes.

TOP

Limitations and Restrictions

Limitations and restrictions on the SQL Server PDW tempdb database. You cannot:

  • Create a global temporary table that begins with ##.

  • Perform a backup or restore of tempdb.

  • Modify permissions to tempdb with the GRANT, DENY, or REVOKE statements.

  • Execute DBCC SHRINKLOG for tempdbtempdb.

  • Perform DDL operations on tempdb. There are a couple exceptions to this. For details, see the following list of limitations and restrictions on local temporary tables.

Limitations and restrictions on local temporary tables. You cannot:

  • Rename a temporary table

  • Create partitions, views, or non-clustered indexes on a temporary table. ALTER INDEX can be used to rebuild a clustered index for a table created with one.

  • Modify permissions to temporary tables with the GRANT, DENY, or REVOKE statements.

  • Run database console commands on temporary tables.

  • Use the same name for two or more temporary tables within the same batch. If more than one local temporary table is used within a batch, each must have a unique name. If multiple sessions are running the same batch and creating the same local temporary table, SQL Server PDW internally appends a numeric suffix to the local temporary table name to maintain a unique name for each local temporary table.

Note Note

You can create and update statistics on a temporary table. ALTER INDEX can be used to rebuild a clustered index.

TOP

Permissions

Any user can create temporary objects in tempdb. Users can only access their own objects, unless they receive additional permissions. It is possible to revoke the connect permission to tempdb to prevent a user from using tempdb, but this is not recommended as some routine operations require the use of tempdb.

Related Tasks

Tasks | Description
Create a table in tempdb. | You can create a user temporary table with the CREATE TABLE and CREATE TABLE AS SELECT statements. For more information, see CREATE TABLE (SQL Server PDW) and CREATE TABLE AS SELECT (SQL Server PDW).
View a list of existing tables in tempdb. | SELECT * FROM tempdb.sys.tables;
View a list of existing columns in tempdb. | SELECT * FROM tempdb.sys.columns;
View a list of existing objects in tempdb. | SELECT * FROM tempdb.sys.objects;

TOP

See Also


master Database (SQL Server PDW)

[image]

master Database (SQL Server PDW)
See Also Send Feedback

The SQL Server PDW master database stores appliance-level login information and the database catalog. It is a SQL Server master database that resides on the Control node. As such, it provides similar functionality to SQL Server PDW as master provides to SQL Server.

For more information about system databases, see System Databases (SQL Server PDW)

Limitations and Restrictions

The following list describes the operations you cannot perform on the SQL Server PDW master database.

You cannot:

  • Perform a differential backup of master.

  • Create user objects in master.

  • Change the collation of master.

  • Change the owner of master.

  • Drop or rename master.

  • Modify permissions to master.

  • Execute DBCC SHRINKLOG.

Related Tasks

Task | Description
Create a full backup of master. | Example: BACKUP DATABASE master TO backup_directory; For more information, see BACKUP DATABASE (SQL Server PDW).
Restore the master database | To restore the master database, use the Restore the Master Database (Analytics Platform System) page in the Configuration Manager tool.
View database catalog information. | SELECT * FROM master.sys.databases;
View system-wide login and permission information. | SELECT * FROM master.sys.server_permissions; SELECT * FROM master.sys.server_principals; SELECT * FROM master.sys.sql_logins;

See Also


User Sessions (SQL Server PDW)

[image]

User Sessions (SQL Server PDW)
See Also Send Feedback

A login with the appropriate permissions can manage the sessions of all logins on a SQL Server PDW appliance, including performing these actions:

  • View the current sessions on the appliance, including both active and idle sessions.

  • View the active and recent queries for a session.

  • End active sessions.

These actions can be performed by using either the Monitor the Appliance by Using the Admin Console (Analytics Platform System) or System Views (SQL Server PDW) through SQL commands, as shown below.

The permissions required to manage sessions by using either method are the same, and are described in Grant Permissions to Manage Logins, Users, and Database Roles (SQL Server PDW).

Manage Sessions by Using the Admin Console

To view current sessions by using the Admin Console

  1. On the top menu, click Sessions.

  2. The resulting list displays all recent sessions. To view only ‘Active’ or ‘Idle’ sessions, click the Status column header to sort results by status.

To view active and recent queries for a session by using the Admin Console

  1. On the top menu, click Sessions.

  2. In the results list, click the session ID of the desired session.

  3. The resulting queries list shows the recent queries for the session. For information on viewing query details, see Monitoring Active Queries (SQL Server PDW).

To end sessions by using the Admin Console

  1. On the top menu, click Sessions.

  2. Find the session ID for the session to cancel.

  3. Click the red X to the left of the session ID to end the session. Only sessions with a status of ‘Active’ or ‘Idle’ will have a red X; only these sessions can be ended.

Manage Sessions by Using System Views and SQL Commands

To View Current Sessions by Using System Views

Use sys.dm_pdw_exec_sessions (SQL Server PDW) to generate a list of current sessions.

This example returns the session_id, login_name, and status for all sessions with a status of ‘Active’ or ‘Idle’.

Copy image Copy Code
SELECT session_id, login_name, status FROM sys.dm_pdw_exec_sessions WHERE status='Active' OR status='Idle';

To View Active and Recent Queries for a Session by Using System Views

To see the active and recently completed queries associated with a session, you use the sys.dm_pdw_exec_sessions (SQL Server PDW) and sys.dm_pdw_exec_requests (SQL Server PDW) views. This query returns a list of all active or idle sessions, plus any active or recent queries associated with each session ID.

Copy image Copy Code
SELECT es.session_id, es.login_name, es.status AS sessionStatus,
er.request_id, er.status AS requestStatus, er.command
FROM sys.dm_pdw_exec_sessions es
LEFT OUTER JOIN sys.dm_pdw_exec_requests er
ON (es.session_id=er.session_id)
WHERE (es.status='Active' OR es.status='Idle') AND
(er.status!= 'Completed' AND er.status!= 'Failed' AND er.status!= 'Cancelled');

To End Sessions by Using SQL Commands

Use the KILL (SQL Server PDW) command to end a current session. You will need the session ID for the process to terminate, which can be obtained using the sys.dm_pdw_exec_sessions (SQL Server PDW) view.

In this example, select the login_name, session_id, and status values to find a session based on the login name.

Copy image Copy Code
SELECT session_id, login_name, status FROM sys.dm_pdw_exec_sessions;

Sessions with an ‘Active’ or ‘Idle’ status can be ended by using the KILL command.

Copy image Copy Code
KILL 'SID137';

See Also


Workload Management (SQL Server PDW)

[image]

Workload Management (SQL Server PDW)
See Also Send Feedback

SQL Server PDW’s workload management capabilities allow users and administrators to assign requests to pre-set configurations of memory, and concurrency. Use workload management to improve performance of your workload, either consistent or mixed, by allowing requests to have the appropriate resources without starving any requests forever.

For example, with the workload management techniques in SQL Server PDW, you could:

  • Allocate a large number of resources to a load job.

  • Specify more resources for building a columnstore index.

  • Troubleshoot a slow-performing hash join to see if it needs more memory, and then give it more memory.

Contents

  • Workload Management Basics

  • Resource Class Descriptions

  • General Remarks

  • Limitations and Restrictions

  • Metadata

  • Related Tasks

Workload Management Basics

Key Terms

Workload Management

Workload Management is the ability to understand and adjust system resource utilization in order to achieve the best performance for concurrent requests.

Resource Class

In SQL Server PDW, a resource class is a built-in server role that has pre-assigned limits for memory and concurrency. SQL Server PDW allocates resources to requests according to the resource class server role membership of the login that submits the requests.

On the Compute nodes, the implementation of resource classes uses the Resource Governor feature in SQL Server. For more information about Resource Governor, see Resource Governor on MSDN.

Understand Current Resource Utilization

To understand system resource utilization for the currently running requests, use the SQL Server PDW dynamic management views. For example, you can use DMVs to understand if a slow-running large hash join could benefit by having more memory.

For examples, see Common Metadata Query Examples (SQL Server PDW).

Adjust Resource Allocations

To adjust resource utilization, change the resource class membership of the login that is submitting the request. The resource class server roles are named mediumrc, largerc, and xlargerc. They represent medium, large, and extra large resource allocations respectively.

For example, to allocate a large amount of system resources to a request, add the login that is submitting the request to the largerc server role. The following ALTER SERVER ROLE statement adds the login Anna to the largerc server role.

Copy image Copy Code
ALTER SERVER ROLE largerc ADD MEMBER Anna;

Resource Class Descriptions

The following table describes the resource classes and their system resource allocations.

Resource Class

Request Importance

Maximum Memory Usage*

Concurrency Slots (Maximum = 32)

Description

default

Medium

400 MB

1

By default, each login is allowed a small amount of memory, and concurrency resources for its requests.

When a login is added to a resource class, the new class takes precedence. When a login is dropped from all resource classes, the login reverts back to the default resource allocation.

MediumRC

Medium

1200 MB

3

Examples of requests that might need the medium resource class:

  • CTAS operations that have large hash joins.

  • SELECT operations that need more memory to avoid caching to disk.

  • Loading data into clustered columnstore indexes.

  • Building, rebuilding, and reorganizing clustered columnstore indexes for smaller tables that have 10-15 columns.

Largerc

High

2.8 GB

7

Examples of requests that might need the large resource class:

  • Very large CTAS operations that have huge hash joins, or contain large aggregations, such as large ORDER BY or GROUP BY clauses.

  • SELECT operations that require very large amounts of memory for operations such as hash joins, or aggregations such as ORDER BY or GROUP BY clauses

  • Loading data into clustered columnstore indexes.

  • Building, rebuilding, and reorganizing clustered columnstore indexes for smaller tables that have 10-15 columns.

xlargerc

High

8.4 GB

22

The extra large resource class is for requests that could require extra large resource consumption at run time.

*Maximum memory usage is an approximation.

Request Importance

The request importance maps to the amount of CPU time that SQL Server, running on the Compute nodes, will give to the requests. Requests with higher priority receive more CPU time.

Maximum Memory Usage

Maximum memory usage is the maximum amount of available memory a request can use within each processing space. For example a mediumrc request can use up to 1200 MB for processing within each distribution. It is still important to ensure data is not skewed in order to avoid having a few distributions performing most of the work.

Concurrency Slots

The goal of allocating 1, 3, 7, and 22 concurrency slots is to allow both large and small processes to run at the same time, without blocking small process when a large process is running. For example, SQL Server PDW can allocate maximum of 32 concurrency slots to run 1 extra large request (22 slots), 1 large request (7 slots), and 1 medium request (3 slots) at the same time.

Examples of allocating up to 32 concurrency slots to concurrent requests:

  • 28 slots = 4 large

  • 30 slots = 10 medium

  • 32 slots = 32 default

  • 32 slots = 1 extra large + 1 large + 1 medium

  • 32 slots = 2 large + 4 medium + 6 default

Suppose 6 large requests are submitted to SQL Server PDW, and then 10 default requests are submitted. SQL Server PDW will process the requests in priority order as follows:

  • Allocate 28 concurrency slots to start processing 4 large requests as memory becomes available, and keep 2 large requests in the queue.

  • Allocate 4 concurrency slots to start processing 4 default requests and keep 6 default requests in the wait queue.

As requests finish and concurrency slots become available, SQL Server PDW will allocate the remaining requests according to available resources and priority. For example, when there are 7 concurrency slots open, waiting large requests will have higher priority for the 7 slots than waiting medium requests. If 6 slots open, then SQL Server PDW will allocate 6 more default-sized requests. However, memory and concurrency slots must all be available before SQL Server PDW allows a request to run.

Within each resource class, the requests run in first in first out (FIFO) order.

General Remarks

If a login is a member of more than one resource class, the class with the most resources takes precedence.

When a login is added to or dropped from a resource class, the change takes effect immediately for all future requests; current requests that are running or waiting are not affected. The login does not need to disconnect and reconnect in order for the change to occur.

For each login, the resource class settings are applied to individual statements and operations, and not to the session.

Before SQL Server PDW runs a statement, it tries to acquire the concurrency slots needed for the request. If it cannot acquire enough concurrency slots, SQL Server PDW moves the request into a waiting-to-be-executed state. All resources system that were already allocated to the request are returned back to the system.

Most of the SQL statements always need the default resource allocations, and therefore are not controlled by resource classes. For example, CREATE LOGIN only needs a small amount of resources, and is allocated the default resources even if the login calling CREATE LOGIN is a member of a a resource class. For example, if Anna is a member of the largerc resource class and she submits a CREATE LOGIN statement, the CREATE LOGIN statement will run with the default number of resources.

SQL statements and operations governed by resource classes:

  • ALTER INDEX REBUILD

  • ALTER INDEX REORGANIZE

  • ALTER TABLE REBUILD

  • CREATE CLUSTERED INDEX

  • CREATE CLUSTERED COLUMNSTORE INDEX

  • CREATE TABLE AS SELECT

  • CREATE REMOTE TABLE AS SELECT

  • Loading data with dwloader.

  • INSERT-SELECT

  • UPDATE

  • DELETE

  • RESTORE DATABASE when restoring into an appliance with more Compute nodes.

  • SELECT, excluding DMV-only queries

Limitations and Restrictions

The resource classes govern memory and concurrency allocations. They do not govern input/output operations.

Metadata

DMVs that contain information about resource classes and resource class members.

  • sys.server_role_members (SQL Server PDW)

  • sys.server_principals (SQL Server PDW)

DMVs that contain information about the state of requests and the resources they require:

  • sys.dm_pdw_lock_waits (SQL Server PDW)

  • sys.dm_pdw_resource_waits (SQL Server PDW)

Related system views exposed from the SQL Server DMVs on the Compute nodes. See SQL Server Dynamic Management Views (SQL Server PDW) for links to these DMVs on MSDN.

  • sys.dm_pdw_nodes_resource_governor_resource_pools

  • sys.dm_pdw_nodes_resource_governor_workload_groups

  • sys.dm_pdw_nodes_resource_governor_resource_pools

  • sys.dm_pdw_nodws_resource_governor_workload_groups

  • sys.dm_pdw_nodes_exec_sessions

  • sys.dm_pdw_nodes_exec_requests

  • sys.dm_pdw_nodes_exec_query_memory_grants

  • sys.dm_pdw_nodes_exec_query_resource_semaphores

  • sys.dm_pdw_nodes_os_memory_brokers

  • sys.dm_pdw_nodes_os_memory_cache_entries

  • sys.dm_pdw_nodes_exec_cached_plans

Related Tasks

View Login Members of Each Resource Class (SQL Server PDW)

Change the System Resources Allocated to a Request (SQL Server PDW)

Change a login to the default system resources for its requests

Display the Number of Concurrency Slots Needed for a Waiting Request (SQL Server PDW)

See the Workload Management section of Common Metadata Query Examples (SQL Server PDW) for the following tasks:

  1. Find the number of active requests for each resource group.

  2. Determine if my requests need more memory

  3. Determine if there are a high number of query optimizations or suboptimal plan generations.

  4. Determine Average CPU time per request in each resource pool to date

See Also


View Login Members of Each Resource Class (SQL Server PDW)

[image]

View Login Members of Each Resource Class (SQL Server PDW)
Send Feedback

Describes how to display the login members of each resource class server role in SQL Server PDW. Use this query to figure out the class of resources allowed for requests submitted by each login.

For resource class descriptions, see Workload Management (SQL Server PDW).

View Login Members of Each Resource Class

This query displays the membership list for each resource class. There are three resource classes, mediumrc, largerc, and xlargerc.

Copy image Copy Code
SELECT l.name AS [member], r.name AS [server role]
FROM sys.server_role_members AS rm
JOIN sys.server_principals AS l
  ON l.principal_id = rm.member_principal_id
JOIN
  sys.server_principals AS r
  ON r.principal_id = rm.role_principal_id
WHERE
  l.[type] = 'S'
  AND r.[type] = 'R'
  AND r.[name] in ('mediumrc', 'largerc', 'xlargerc');

If a login is not in this list, its requests will receive the default resources. If a login is a member of more than one resource class, the largest class has precedence.

The resource allocations are listed in the Workload Management (SQL Server PDW) topic.


Change the System Resources Allocated to a Request (SQL Server PDW)

[image]

Change the System Resources Allocated to a Request (SQL Server PDW)
See Also Send Feedback

Describes how to figure out which resource class a SQL Server PDW request is running under, and then how to change the system resources for that request. Changing the resources for a request requires changing the resource class membership of the login submitting the request, by using the ALTER SERVER ROLE (SQL Server PDW) statement.

Step 1: Determine the resource class for the login running the request.

This query displays logins which are members of the resource class server role memberships. There are three resource classes, mediumrc, largerc, and xlargerc.

Security note Security Note

This query must be executed by a login having CONTROL SERVER permission. If executed by a login without CONTROL SERVER permission, this query only returns the role memberships for the current login.

Copy image Copy Code
SELECT l.name AS [member], r.name AS [server role]
FROM sys.server_role_members AS rm
JOIN sys.server_principals AS l
  ON l.principal_id = rm.member_principal_id
JOIN
  sys.server_principals AS r
  ON r.principal_id = rm.role_principal_id
WHERE
  l.[type] = 'S'
  AND r.[type] = 'R'
  AND r.[name] in ('mediumrc', 'largerc', 'xlargerc');
GO

If there are no logins that are members of a resource class server role, the resulting table will be empty. In this case, if the query returns a login named Ching, then when Ching submits a request, the request will receive the default system resources, which is smaller than the resource class system resources. If a login is a member of more than one resource class, the largest class has precedence.

For a list of resource allocations for each resource class, see Workload Management (SQL Server PDW) topic.

Step 2: Run the request under a login with different resource class membership

There are two ways to run a request with either larger or smaller system resources:

  • Run the request under a different login that is a member of a larger or smaller resource class.

  • Add the necessary login to one of the resource class roles. Choose this option with caution; changing the resource class for the login will change the system resource level for all requests submitted by the login.

Suppose Ching is a member of the largerc server role. The following example shows how to add login Ching to the xlargerc server role.

Copy image Copy Code
ALTER SERVER ROLE xlargerc ADD MEMBER Ching;

Ching is now a member of the largerc and the xlargerc server roles. When Ching submits requests, the requests will receive the xlargerc system resources.

The following example moves Ching back to the mediumrc server role. To do this, she must be removed from xlargerc, and largerc server roles, and added to the mediumrc server role.

Copy image Copy Code
-- Move login Ching back to using medium system resources for requests.
ALTER SERVER ROLE xlargerc DROP MEMBER Ching;
ALTER SERVER ROLE largerc DROP MEMBER Ching;
ALTER SERVER ROLE mediumrc ADD MEMBER Ching;

Ching is now a member of the mediumrc server role. The following example changes Ching to have the default system resources for her requests.

Copy image Copy Code
-- Move login Ching to use the default system resources for requests.
ALTER SERVER ROLE mediumrc DROP MEMBER Ching;

For more information about changing resource class role membership, see ALTER SERVER ROLE (SQL Server PDW).

See Also

Concepts

Workload Management (SQL Server PDW)


Change a login to the default system resources for its requests

[image]

Change a login to the default system resources for its requests
Send Feedback

Describes how to change the system resource allocations assigned to a SQL Server PDW login to the default amounts. This affects the system resources that SQL Server PDW assigns to requests submitted by the login.

For resource class descriptions, see Workload Management (SQL Server PDW)

Use the default level of system resources

When a login is not a member of any resource class server role, requests submitted by the login will receive the default amount of system resources.

Suppose the login Matt is currently a member of all resource class server roles and wants to revert back to having his requests receive only the default resources. The following example assigns the default resources to Matt’s requests by dropping his membership from all three resource class server roles.

Copy image Copy Code
--Give the requests submitted by Matt the default system resources
--by dropping Matt from all resource class server roles.
ALTER SERVER ROLE XLargeRC DROP MEMBER Matt;
ALTER SERVER ROLE LargeRC DROP MEMBER Matt;
ALTER SERVER ROLE MediumRC DROP MEMBER Matt;


Display the Number of Concurrency Slots Needed for a Waiting Request (SQL Server PDW)

[image]

Display the Number of Concurrency Slots Needed for a Waiting Request (SQL Server PDW)
Send Feedback

Describes how to figure out the number of concurrency slots are needed by a request that is waiting to run on SQL Server PDW.

For more information, see Workload Management (SQL Server PDW)

Display the Number of Concurrency Slots Needed for a Waiting Request

A request could be waiting too long without getting executed. One of the ways to troubleshoot this is to look at the number of concurrency slots the request needs. The following example shows the number of concurrency slots needed by each waiting request.

Copy image Copy Code
--Display the number of concurrency slots required
--for each request that is waiting to run.
SELECT request_id, concurrency_slots_used AS [Slots Needed], resource_class AS [Resource Class]
FROM sys.dm_pdw_resource_waits;


SQL Reference (SQL Server PDW)

[image]

SQL Reference (SQL Server PDW)
See Also Send Feedback

SQL is central to using SQL Server PDW. All applications that communicate with an instance of SQL Server PDW do so by sending SQL statements to the appliance, regardless of the user interface of the application.

SQL Reference Topics

The SQL reference topics are categorized into the following sections.

  • Language Elements (SQL Server PDW)

  • Data Definition Statements (SQL Server PDW)

  • Data Manipulation Statements (SQL Server PDW)

  • Database Console Commands (SQL Server PDW)

  • Queries (SQL Server PDW)

  • CREATE DIAGNOSTICS SESSION (SQL Server PDW)

  • Transactions (SQL Server PDW)

To view the individual language components such as operators, statements, functions, data types, and more, you can browse through the table of contents or search for topics in the index.

See Also


Language Elements (SQL Server PDW)

[image]

Language Elements (SQL Server PDW)
See Also Send Feedback

SQL Server PDW includes a number of optional or required syntactic elements that are used to construct statements.

In This Section

Topic | Description
Syntax Conventions | Describes the conventions used in the syntax diagrams in the SQL Reference.
Object Naming Rules | Describes the rules for creating names for objects: databases, tables, views, indexes, roles, users, and columns.
Reserved Keywords | Lists words that cannot be for user-created table names, view names, column names, or other user-specified names.
Collations (SQL Server PDW) | Describes the rules for how strings of character data are sorted and compared.
Constants | Describes how to format literal or scalar values, depending on the data type the value represents.
Data Types | Describes the types of data that objects can hold. For example, integer data, character data, monetary data, date and time data, and so on.
Expressions | Describes the terminology and syntax rules for expressions which are used in SQL statements.
Operators | Describes and links to the operator definitions by category. You can also find individual operator definitions by browsing the Table of Contents under this topic, or by using the index.
Functions | Describes and links to the functions available in SQL Server PDW.
Locking Behavior | Describes the types of locks that SQL Server PDW supports to ensure the integrity of transactions and maintain the consistency of databases when multiple users are accessing data at the same time.

The section also contains syntax for statements that are not categorized in any the groups listed above: Comments, EXECUTE, KILL, PRINT, and USE.

See Also

Reference

Syntax Conventions (SQL Server PDW)


Syntax Conventions (SQL Server PDW)

[image]

Syntax Conventions (SQL Server PDW)
See Also Send Feedback

Describes syntax conventions and multipart name rules that are used in the syntax diagrams in the SQL Reference topics.

Contents

  • Syntax Conventions

  • Multipart Names

Syntax Conventions

The following table lists and describes conventions that are used in the syntax diagrams in the SQL Reference.

Convention

Used For

UPPERCASE

SQL keywords.

italic

User-supplied parameters of SQL syntax.

bold

Database names, table names, column names, index names, data type names, and text that must be typed exactly as shown.

underline

Indicates the default value applied when the clause that contains the underlined value is omitted from the statement.

| (vertical bar)

Separates alternative syntax items enclosed in brackets or braces. You can use only one of the items.

[ ] (brackets)

Optional syntax items. Do not type the brackets.

{ } (braces)

Required syntax items. Do not type the braces.

[,...n]

Indicates that the preceding item can be repeated n number of times. The occurrences are separated by commas.

[...n]

Indicates the preceding item can be repeated n number of times. The occurrences are separated by blanks.

;

SQL statement terminator. The semicolon is optional except to separate statements within multi-statement transactions.

<label> ::=

The name for a block of syntax. This convention is used to group and label sections of lengthy syntax or a unit of syntax that can be used in more than one location within a statement. Each location in which the block of syntax can be used is indicated with the label enclosed in angle brackets: <label>.

A set is a collection of expressions, for example <grouping set>; and a list is a collection of sets, for example <composite element list>.

Multipart Names

Unless specified otherwise, all references to the name of a SQL Server PDW database object can be a three-part name:

database_name.schema_name.object_name

The name can be referenced using any of the following:

object_name | database_name..object_name | schema_name.object_name | database_name.schema_name.object_name

database_name

Specifies the name of a SQL Server PDW database.

schema_name

Refers to the name of the current schema.

object_name

Refers to the name of the object.

See Also


Object Naming Rules (SQL Server PDW)

[image]

Object Naming Rules (SQL Server PDW)
See Also Send Feedback

This topic describes the permitted names for backup directory names and SQL Server PDW objects, including databases, tables, views, indexes, columns, statistics, logins, and server roles.

Object names can be of two types: standard object names that comply with SQL Server PDW object naming rules, and delimited object names that can violate SQL Server PDW object naming rules if they are enclosed by delimiting characters.

Object Naming Rules

SQL Server PDW object names can include the following characters

  1. Letters, as defined by Unicode Standard 3.2. The Unicode definition of letters includes Latin characters from a-z, A-Z, and letter characters from other languages.

  2. Numeric characters 0-9.

  3. Special characters _, @, $, #.

SQL Server PDW object names must also meet the following requirements:

  1. Object names must not be Transact-SQL or SQL Server PDW reserved words. SQL Server PDW reserves the uppercase, lowercase, and mixed-case versions of reserved words. See Transact-SQL Reserved Words and Reserved Keywords (SQL Server PDW).

  2. Embedded spaces are not allowed in object names. (In most cases, leading and trailing spaces do not cause errors but are automatically removed, unless they are Delimited Object Names.)

  3. Unicode supplementary characters are not allowed in object names.

For each object type, additional restrictions apply.

  • Database, table, and view names must be 1-128 characters long and begin with a letter character. They are case insensitive. Database names are unique per appliance. Table and view names are unique per database.

    A local temporary table name must begin with #, and cannot exceed 116 characters including the #.

  • Index, column, and statistics names must be 1-32 characters long and begin with a letter character. They are case insensitive. Index names are unique per database. Column and statistics names are unique per table.

  • Login and server role names must be 4-128 characters long and begin with a letter character. They are case insensitive and unique per appliance.

  • Backup directory names must be 1-128 characters long and begin with a letter or number character or an underscore (_). Special characters permitted are the underscore (_), hyphen (-), or space ( ). Backup names cannot end with a space character.

Delimited Object Names

Object names can violate the object naming rules if delimiting characters are used to encapsulate the object name in all SQL commands.

Special characters not supported with regular object names can be included in delimited object names. They include space ( ), tilde (~), exclamation point (!), percent (%), caret (^), ampersand (&), left parentheses ((), right parenthesis ()), hyphen (-), left brace ({), right brace (}), apostrophe ('), period (.), backslash (\), and accent grave (`). Delimited object names can also be reserved words, such as “order.”

Delimited object names cannot violate the length and uniqueness requirements for object names.

The delimiting characters are the double quotation marks (") and square brackets ([]). The following example shows the use of delimiting square brackets.

Copy image Copy Code
SELECT *
FROM [My Table]      --Identifier contains a space and uses a reserved keyword.
WHERE [order] = 10   --Identifier is a reserved keyword.

The following examples use bracketed identifiers for table names and column names in a CREATE TABLE statement.

Copy image Copy Code
CREATE TABLE [^$Employee Data]

(

 [^First Name]   varchar(25) NOT NULL,

 [^Last Name]   varchar(25) NOT NULL,

 [^Dept ID]   int

);

A SELECT statement can be written as shown below to retrieve rows from the above table.

Copy image Copy Code
SELECT * FROM [^$Employee Data];

The following examples use quoted identifiers for table names and column names.

Copy image Copy Code
CREATE TABLE "$Employee Data"
(
 "^First Name"   varchar(25) NOT NULL,
 "^Last Name"   varchar(25) NOT NULL,
 "^Dept ID"   int
);

To return the data from "$Employee Data" table, the following syntax can be used.

Copy image Copy Code
SELECT * FROM "$Employee Data"

The following example creates a a table named table with columns for tablename, user, select, insert, update, and delete. Because TABLE, SELECT, INSERT, UPDATE, and DELETE are reserved keywords, the identifiers must be delimited every time the objects are accessed.

Copy image Copy Code
CREATE TABLE [table]
(
 tablename char(128) NOT NULL,
 [USER]    char(128) NOT NULL,
 [SELECT]  char(128) NOT NULL,
 [INSERT]  char(128) NOT NULL,
 [UPDATE]  char(128) NOT NULL,
 [DELETE]  char(128) NOT NULL
);

Delimiting Identifiers with Multiple Parts

When you are using qualified object names, you may have to delimit more than one of the identifiers that make up the object name. Each identifier must be delimited individually.

Copy image Copy Code
SELECT * FROM [My DB].[dbo].[My.Table];

Or

SELECT * FROM "My DB"."My#UserID"."My.Table";

Guidelines for Delimited Identifiers

When using delimited object names, keep the following guidelines in mind.

  1. Both leading and trailing spaces are supported in delimited object names. However, using them may cause unexpected results when comparing against values without leading or trailing spaces. For this reason, using object names with leading or trailing spaces may not be prudent on some systems.

  2. Although SQL Server PDW supports the use of reserved words as object names by using a delimiter, this practice is not recommended.

  3. SQL Server PDW allows any character in the current code page to be used in a delimited identifier. However, indiscriminate use of special characters in an object name may make SQL statements and scripts difficult to read and maintain. For example, you can create a table with the name Employee], where the closing square bracket is part of the name. To do this, you have to escape the closing square bracket using two more square brackets as shown in the following:

    Copy image Copy Code
    CREATE TABLE [Employee]]]
(
EmployeeID int,
FirstName varchar(30),
LastName varchar(30)
);

  4. Double quotation marks can only be used to delimit identifiers. They cannot be used to delimit character strings. NOTE: Single quotation marks must be used to enclose character strings. They cannot be used to delimit identifiers. If the character string contains an embedded single quotation mark, you should insert an additional single quotation mark in front of the embedded mark as shown in the following example.

    Copy image Copy Code
    SELECT * FROM MyTable
WHERE LastName = 'O''Brien';

See Also

Reference

CREATE DATABASE (SQL Server PDW)
CREATE TABLE (SQL Server PDW)
CREATE VIEW (SQL Server PDW)
CREATE INDEX (SQL Server PDW)
CREATE STATISTICS (SQL Server PDW)
CREATE LOGIN (SQL Server PDW)
CREATE SERVER ROLE (SQL Server PDW)


Reserved Keywords (SQL Server PDW)

[image]

Reserved Keywords (SQL Server PDW)
See Also Send Feedback

Microsoft SQL Server PDW uses reserved keywords for defining, manipulating, and accessing databases. Reserved keywords are part of the grammar of the SQL Server PDW language that is used by SQL Server PDW to parse and understand SQL Server PDW statements and batches. Although not enumerated below, all SQL Server Transact-SQL reserved keywords are also reserved in SQL Server PDW. (See Reserved Keywords (Transact-SQL) on MSDN for a list of the Transact-SQL reserved keywords.)

The following table lists SQL Server PDW reserved keywords.

ABS | COPY | NTILE
ACOS | COS | NULLIFZERO
ADD_MONTHS | DATE_TRUNC | NVL
ADMIN | DATE_PART | PAGE
AFTER | DATEPART | PAGE_SIZE
AGE | DECODE | PARTITION
AGGREGATE | DENSE_RANK | PARTITIONED
ALIAS | DISKUSAGE | PARTITIONING
ARRAY | DISTRIBUTE | PARTITIONS
ASIN | DISTRIBUTE_ON | PARTITION_ON
ASSIGNMENT | DISTRIBUTED_SIZE | PATINDEX
ASYMMETRIC | DISTRIBUTION_ON | PASSWORD
ATAN | EACH | QUERY
ATTRIBUTES | ENCRYPTION | RANGE
AUTOGROW | EXP | RANK
AUTOMATIC | EXPLAIN | REPLACE
AVG | EXPORT | REPLICATE
BEFORE | EXTEND | REPLICATED_SIZE
BINAY | FAST_INSERT | ROLE
BITVAR | FLOAT | ROUND
BOOLEAN | FLOAT8 | ROW_ALLOCATION
BYTE | FLOOR | ROW_COUNT
CATALOG_NAME | FORMAT | ROW_NUMBER
CEIL | GREATER | RPAD
CEILING | GREATEST | RTRIM
CHAIN | HISTORY | SAMPLE
CHAR_LENGTH | IFNULL | SAVEPOINT
CHARACTER_LENGTH | IIPARTITION | SESSION_ID
CHARACTER_SET_CATALOG | INT1 | SHOW
CHARACTER_SET_NAME | INT2 | SIGN
CHARACTER_SET_SCHEMA | INT4 | SIN
CHARINDEX | INT8 | SMALLER
CHECKED | ISNUMERIC | SQRT
CLASS_ORIGIN | ISO-WEEK | STATISTICS
CLUSTER_ON | LABEL | STDDEV_POP
COBOL | LAG | STDDEV_SAMP
COLLATION_CATALOG | LEAD | STDEVP
COLLATION_NAME | LEAST | STDEV
COLLATION_SCHEMA | LEN | SUBPARTITION
COLUMN_NAME | LENGTH | SYSDATETIME
COMMAND_FUNCTION | LIMIT | TAN
COMMAND_FUNCTION_NAME | LOCATE | TEXT
COMMITTED | LOCK | THAN
COMPLETE | LOG_SIZE | VARBINARY
COMPRESSED | LOG | VAR
CONDITION_NUMBER | LOG10 | VARP
CONNECTION_NAME | LPAD | VERSION
CONSTRAINT_CATALOG | LTRIM | VIEWS
CONSTRAINT_NAME | MOD | ZEROIFNULL

Additionally, the ISO standard defines a list of reserved keywords. Avoid using ISO reserved keywords for object names and identifiers.

See Also

Reference

Syntax Conventions (SQL Server PDW)


Collations (SQL Server PDW)

[image]

Collations (SQL Server PDW)
See Also Send Feedback

A collation specifies the rules for how strings of character data are sorted and compared, based on the norms of particular languages and locales. Use collations in SQL Server PDW to internationalize strings, so that sorting and comparing strings fit the language and locale of the database user.

Contents

  • Understanding Collations in SQL Server PDW

    • Overview

    • Key Terms

    • Differences between SQL Server PDW and SQL Server Collations

  • Examples

  • Related Collation Tasks

Understanding Collations in SQL Server PDW

Overview

SQL Server PDW has a case insensitive appliance-level collation, Latin1_General_100_CI_AS_KS_WS, which is used for both object names and string data.

For object names, the appliance-level collation is fixed and cannot be changed. For example, login names, database names, and database object names, are always Latin1_General_100_CI_AS_KS_WS.

For string data, the appliance-level collation is the default. However, there is a COLLATE clause that allows you to change the collation of columns and expressions to one of the Windows collations supported by SQL Server. You can specify the collation for a table column by using the COLLATE clause in the CREATE TABLE (SQL Server PDW) statement or in the SELECT portion of the CREATE TABLE AS SELECT (SQL Server PDW) statement. You can also apply the COLLATE clause to any scalar Expressions (SQL Server PDW) that evaluates to a string data type.

Key Terms

collation

Specifies the rules for how strings of character data are sorted and compared, based on the norms of particular languages and locales. For example, in an ORDER BY clause, an English speaker would expect the character string 'Chiapas' to come before 'Colima' in ascending order. However, a Spanish speaker in Mexico might expect words beginning with 'Ch' to appear at the end of a list of words starting with 'C'. Collations dictate these kinds of sorting and comparison rules. The Latin_1 General collation will sort 'Chiapas' before 'Colima' in an ORDER BY ASC clause, whereas the Traditional_Spanish collation will sort 'Chiapas' after 'Colima'.

appliance-level collation

Specifies the collation that is applied to all metadata and databases in the SQL Server PDW appliance. When a database is created, the default collation is the appliance-level collation.

Specify Collations in SQL Server PDW

The following table describes how collations are specified in SQL Server PDW.

Collation Level

SQL Server PDW Description

Appliance

Appliance-level collation is Latin1_General_100_CI_AS_KS_WS. This is pre-configured and cannot be changed. This collation is case-insensitive. All system-wide metadata, login names and database names use this collation and are case-insensitive.

Database

The database collation is set to the appliance-level collation and cannot be changed. By default, all table names, view names, column names, variable names, expressions, string constants, and identifiers use the database-level collation and are case-insensitive.

Column

By default, the column collation is set to the database collation. Column collations only apply to columns of type char, varchar, nchar, or nvarchar.

You can use the COLLATE clause to specify the collation of a string column when the table is created. This is supported in CREATE TABLE and also in the SELECT portion of the CREATE TABLE AS SELECT statement.

Expression

The collation of each expression is determined according to the SQL Server collation precedence rules. For more information, see Collation Precedence on MSDN.

You can use the COLLATE clause to explicitly convert the collation of a scalar expression that evaluates to a string data type. For more information, see Expressions (SQL Server PDW)

Identifier

The collation of an identifier depends on the level at which it is defined. Identifiers of appliance-level objects, such as logins and database names, are assigned the collation of the appliance. Identifiers of objects within a database, such as tables, views, and column names, are assigned the collation of the database, which in this release is equal to the appliance-level collation.

Differences between SQL Server PDW and SQL Server Collations

SQL Server PDW handles collations in the same manner as SMP SQL Server, except for the following differences:

  • SQL Server PDW has an appliance-level collation which is similar in purpose to the SMP SQL Server server collation. In MMP SQL Server PDW, the appliance-level collation is the default collation for all objects and string data in the appliance. In SMP SQL Server, the server collation is the default collation for all objects and string data in the SMP SQL Server instance.

  • In SQL Server PDW, the database-level collation is the same as the appliance-level collation and cannot be changed. In SMP SQL Server, the database collation can be changed by using the COLLATE clause of the ALTER DATABASE statement..

  • SQL Server PDW supports the Windows collations that are supported by SQL Server, except for collations that are deprecated in SQL Server. The deprecated collations are listed in the following table.

    Deprecated Windows Collations

    Suggested Replacement

    Korean_Wansung_Unicode

    Lithuanian_Classic

    Hindi

    Indic_General_90

    Macedonian

    Macedonian_FYROM_90

    Azeri_Latin_90

    Azeri_Latin_100

    Azeri_Cyrilllic_90

    Azeri_Cyrilllic_100

  • SQL Server PDW does not support the SQL Server-only collations that are supported by SQL Server.

  • SQL Server PDW errors when an expression has a no-collation label, whereas SQL Server does not. As a result, SQL Server PDW gives an error when string columns in the same expression have different implicit collations, whereas SQL Server does not. For more information on expressions with a no-collation label, see Collation Precedence (Transact-SQL) on MSDN.

  • Each string column in a table has an implicit collation, which is specified by the CREATE TABLE statement.

    In SQL Server PDW, when string columns do not have matching implicit collations, use the COLLATE clause to explicitly change the collation of any of the column operands. By having at least one explicit COLLATE clause, the expression result will have a collation.

    For example, SQL Server PDW will give an error for the string concatenation operation in the following UPDATE statement. The error occurs because x and y have different implicit collations and + is a collation-insensitive operator.

    Copy image Copy Code
    CREATE TABLE abc
(
x nvarchar(20) COLLATE Corsican_100_CI_AI,
y nvarchar(20) COLLATE Greek_BIN2
)
;
UPDATE abc SET x = x + y;

    In SQL Server PDW, use the COLLATE clause to explicitly change the collation of any of the column operands and thus the collation of the expression result. For example, the following statements will work in SQL Server PDW.

    Copy image Copy Code
    UPDATE abc SET x = x + y COLLATE Corsican_100_CI_AI;

  • SQL Server PDW supports applying a COLLATE clause on an expression that already contains an explicit COLLATE clause. SQL Server gives an error for this. For example, the following statement will work in SQL Server PDW and will not work in SQL Server.

    Copy image Copy Code
    CREATE TABLE SomeTable (id int, name nvarchar(20) COLLATE French_CS_AS);
INSERT INTO SomeTable VALUES (340, N'Francois');

SELECT id FROM SomeTable
WHERE name = (N‘Francois’COLLATE French_CI_AS) COLLATE French_CS_AS;
;

  • In SQL Server PDW, local variables of sql_variant type always have the appliance-level collation, which is case-insensitive. This overrides an assigned collation when the variables are declared. Both of the following examples return 1.

    Copy image Copy Code
    --@var1 and @var2 actually have the PDW-wide
--collation Latin1_General_100_CI_AS_KS_WS
DECLARE @var1 sql_variant = 'a' COLLATE Latin1_General_CS_AS;
DECLARE @var2 sql_variant = 'A' COLLATE Latin1_General_CS_AS
IF @var1 = @var2
    SELECT 1;
ELSE
    SELECT 2

DECLARE @var1 sql_variant = 'a' COLLATE Corsican_100_CI_AI;
DECLARE @var2 sql_variant = 'A' COLLATE Latin1_General_CS_AS
IF @var1 = @var2
    SELECT 1;
ELSE
    SELECT 2

For more information about how to work with collations in SQL Server, see Working With Collations on MSDN.

[TOP]

Examples

A. Explicitly convert a string to a collation

Copy image Copy Code
CREATE TABLE abc
(
x nvarchar(20) COLLATE Corsican_100_CI_AI,
y nvarchar(20) COLLATE Greek_BIN2
)
;
UPDATE abc SET x = x + y COLLATE Corsican_100_CI_AI;

B. Use nested COLLATE clauses

Copy image Copy Code
CREATE TABLE SomeTable (id int, name nvarchar(20) COLLATE French_CS_AS);
INSERT INTO SomeTable VALUES (340, N'Francois');

SELECT id FROM SomeTable
WHERE name = (N‘Francois’COLLATE French_CI_AS) COLLATE French_CS_AS;
;

C. Use CREATE TABLE AS SELECT (CTAS) to change collations

This example uses CTAS to change the collation of all the columns in the DatabaseLog table to Latin1_General_100_CI_AS_KS_WS, which is the default collation for the appliance. The example puts the results of the CTAS into the DatabaseLog2 table

Copy image Copy Code
/* CTAS the DatabaseLog table to DatabaseLog2 table with different collation */
USE
    AdventureWorksPDW2012
  ;
If EXISTS (SELECT * from sys.tables t where t.name = 'DatabaseLog2')
DROP TABLE DatabaseLog2;

CREATE TABLE [
    AdventureWorksPDW2012
  ].[dbo].[DatabaseLog2]
WITH ( DISTRIBUTION = REPLICATE )
AS SELECT
    [DatabaseUser] COLLATE Latin1_General_100_CI_AS_KS_WS AS [DatabaseUser],
[Event] COLLATE Latin1_General_100_CI_AS_KS_WS AS [Event],
[Schema] COLLATE Latin1_General_100_CI_AS_KS_WS AS [Schema],
[Object] COLLATE Latin1_General_100_CI_AS_KS_WS AS [Object],
[TSQL] COLLATE Latin1_General_100_CI_AS_KS_WS AS [TSQL]
FROM
    AdventureWorksPDW2012
  .dbo.DatabaseLog;

D. Use ALTER TABLE to change the collation of a column

This example uses ALTER TABLE (SQL Server PDW) to change the collation of the LastName column in the dimEmployee table.

This change affects only the metadata.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
ALTER TABLE AdventureWorksPDW2012.dbo.dimEmployee
    ALTER COLUMN LastName nvarchar(50)
    COLLATE Latin1_General_100_CI_AS_KS_WS;

Related Collation Tasks

Description | Links
Use the COLLATE clause to specify the collation of a column in the CREATE TABLE (SQL Server PDW) statement. | CREATE TABLE (SQL Server PDW)
Use the COLLATE clause to specify the collation of a column in the SELECT portion of the CREATE TABLE AS SELECT statement. | CREATE TABLE AS SELECT (SQL Server PDW)
Use the COLLATE clause to change the collation of a column in an existing table. | ALTER TABLE (SQL Server PDW)

[TOP]

See Also


Comments (SQL Server PDW)

[image]

Comments (SQL Server PDW)
See Also Send Feedback

Examples for creating comments in SQL for SQL Server PDW.

Examples

A. Use two hyphens to comment a line

You can comment a line by starting it with --.

Copy image Copy Code
--View all columns in
    AdventureWorksPDW2012
   having Latin1_General_Bin2 collation
USE
    AdventureWorksPDW2012
  ;
SELECT a.name AS 'Table Name', b.name AS 'Column Name', b.collation_name AS 'Collation Name'
FROM sys.tables a JOIN sys.columns b ON a.object_id=b.object_id
WHERE b.collation_name = 'Latin1_General_Bin2';

B. Use /* and */ to comment a chunk

You can comment a chunk of text by starting it with /* and ending it with */.

Copy image Copy Code
/* Save this code for later use.
USE
    AdventureWorksPDW2012
  ;
SELECT a.name AS 'Table Name', b.name AS 'Column Name', b.collation_name AS 'Collation Name'
FROM sys.tables a JOIN sys.columns b ON a.object_id=b.object_id
WHERE b.collation_name = 'Latin1_General_Bin2';
*/

C. Use nested comments

You can nest /* and */ to create a comment around a chunk that contains a comment.

Copy image Copy Code
/* Don't run this.
/* View all columns in
    AdventureWorksPDW2012
   having Latin1_General_Bin2 collation */
USE
    AdventureWorksPDW2012
  ;
SELECT a.name AS 'Table Name', b.name AS 'Column Name', b.collation_name AS 'Collation Name'
FROM sys.tables a JOIN sys.columns b ON a.object_id=b.object_id
WHERE b.collation_name = 'Latin1_General_Bin2';
*/

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Language Elements (SQL Server PDW)
SQL Reference (SQL Server PDW)


Constants (SQL Server PDW)

[image]

Constants (SQL Server PDW)
See Also Send Feedback

A constant, also known as a literal or a scalar value, is a symbol that represents a specific data value. The format of a constant depends on the data type of the value it represents.

Topic link icon Syntax Conventions (SQL Server PDW)

Character string constants

Character string constants are enclosed in single quotation marks and include alphanumeric characters (a-z, A-Z, and 0-9) and special characters, such as an exclamation point (!), at sign (@), or number sign (#). Character string constants are assigned the default collation.

An extra single quote (') is the syntactic escape character. If a character string enclosed in single quotation marks contains an embedded quotation mark, represent the embedded single quotation mark with two single quotation marks.

The following are examples of character strings:

Copy image Copy Code
'Cincinnati'
'O''Brien'
'Process X is 50% complete.'

Empty strings are represented as two single quotation marks with nothing in between them.

Date and time constants

Date and time constants are represented by using character date values in specific formats, enclosed in single quotation marks. For more information about the formats for date and time constants, see Data Types (SQL Server PDW).

The following are examples of date and time constants:

Copy image Copy Code
'December 5, 1985'
'5 December, 1985'
'851205'
'12/5/98'

Examples of time constants are:

Copy image Copy Code
'14:30:24'
'04:24 PM'
integer constants

integer constants are represented by a string of numbers that are not enclosed in quotation marks and do not contain decimals.

The following are examples of integer constants:

Copy image Copy Code
1894
2
decimal constants

decimal constants are represented by a string of numbers that are not enclosed in quotation marks and contain a decimal point.

The following are examples of decimal constants:

Copy image Copy Code
1894.1204
2.0
float constants

float constants are represented by using scientific notation.

The following are examples of float values:

Copy image Copy Code
101.5E5
0.5E-2

Specifying Negative and Positive Numbers

To indicate whether a number is positive or negative, apply the + or - unary operators to a numeric constant. This creates a numeric expression that represents the signed numeric value. Numeric constants use positive expressions when the + or - unary operators are not applied.

  • Signed integer expressions:

    Copy image Copy Code
    +145345234
-2147483648

  • Signed decimal expressions:

    Copy image Copy Code
    +145345234.2234
-2147483648.10

  • Signed float expressions:

    Copy image Copy Code
    +123E-3
-12E5

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


Data Types (SQL Server PDW)

[image]

Data Types (SQL Server PDW)
See Also Send Feedback

SQL Server PDW data types are a subset of the SQL Server data types. Because SQL Server PDW data is stored and operated on in SQL Server databases, data type definitions and operations are the same as SQL Server, except that SQL Server PDW supports fewer data types than SQL Server.

Contents

  • Data Types

  • Data Type Conversions

Data Types

The following tables list the supported SQL Server PDW data types and provide a link to the SQL Server documentation for each type. The CREATE TABLE (SQL Server PDW) statement gives the syntax and describes how to define each of the SQL Server PDW data types.

Data Type | SQL Server topic | Comments
datetimeoffset | datetimeoffset | For additional information, see Using Date and Time Data.
Datetime2 | datetime2 |
datetime | datetime |
smalldatetime | smalldatetime |
date | date |
time | time |
float and real | float and real | SQL Server PDW does not support the subnormal values: -1.175494351E-038 < value < 0 0 < value < 1.175494351E-038
decimal | decimal |
money and smallmoney | money and smallmoney | When displaying data stored as type money or smallmoney, SQL Server PDW converts the data to the decimal type with a precision and scale of (19, 4) for money and (10, 4) for smallmoney.
int, bigint, smallint, tinyint | int, bigint, smallint, and tinyint |
bit | bit |
nchar and nvarchar | nchar and nvarchar | SQL Server PDW does not support nvarchar (max).
char and varchar | char and varchar | SQL Server PDW does not support varchar (max).
binary and varbinary | binary and varbinary | SQL Server PDW does not support varbinary (max).

Additional Data Types: In addition to the supported data types, the data types nvarchar(max), uniqueidentifier, numeric, and sysname are supported in the SQL Server catalog views only, and may not be used with or for user data.

[TOP]

Data Type Conversion

When using the SELECT (SQL Server PDW) statement, SQL Server PDW supports the SQL Server data type conversion rules for both implicit and explicit data type conversions.

An explicit data type conversions uses the CAST or CONVERT function to perform the conversion. For more information, see CAST and CONVERT (SQL Server PDW). An implicit data type conversion occurs when SQL Server PDW converts one data type to another data type without the CAST or CONVERT function.

When an operator combines two expressions of different data types, the data type with the lower precedence is implicitly converted to the data type with the higher precedence. This applies to supported implicit conversions; if the conversion is not a supported implicit conversion, an error is returned.

For a list of the supported implicit data type conversions in SQL Server, see For more information about data type conversions, see Data Type Conversion (Database Engine) on MSDN.

[TOP]

Data Type Precedence Order for Implicit Conversions

The following list shows the precedence order for implicit conversions, from highest to lowest. For example, datetime2 has higher precedence than datetime. If a value of datetime2 is compared to a value of datetime, the value of datetime will be converted to a datetime2 before SQL Server PDW performs the comparison.

datetimeoffset

datetime2

datetime

smalldatetime

date

time

float

real

decimal

money

smallmoney

bigint

int

smallint

tinyint

bit

nvarchar

nchar

varchar

char

varbinary

binary

[TOP]

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)

Other Resources

Data Types (Transact-SQL)
Data Types (Database Engine)


EXECUTE (SQL Server PDW)

[image]

EXECUTE (SQL Server PDW)
See Also Send Feedback

Executes a stored procedure or character string in SQL Server PDW.

Invocation of a stored procedure without using EXECUTE or EXEC is possible only if the statement is the first statement in the batch.

Security note Security Note

Before you call EXECUTE with a character string, validate the character string. Never execute a command constructed from user input that has not been validated. For more information, see SQL Injection.

Topic link icon Syntax Conventions (SQL Server PDW)Transact-SQL Syntax Conventions

Syntax

Transact-SQL
Execute a stored procedure
[ { EXEC | EXECUTE } ]
    procedure_name
        [ { value | @variable [ OUT | OUTPUT ] } ] [ ,...n ] }
[;]

Execute a SQL string
{ EXEC | EXECUTE }
    ( { @string_variable | [ N ] 'tsql_string' } [ +...n ] )
[;]

Arguments

procedure_name

Is the fully qualified or non-fully qualified name of the stored procedure to call. Procedure names must comply with the rules for identifiers. The names of extended stored procedures are always case-sensitive.

value

Is the value of the parameter to pass to the stored procedure. In SQL Server PDW parameter values must be supplied in the order defined in the procedure.

If the value of a parameter is an object name, character string, or qualified by a database name or schema name, the whole name must be enclosed in single quotation marks. If the value of a parameter is a keyword, the keyword must be enclosed in double quotation marks.

The default can also be NULL. Generally, the procedure definition specifies the action that should be taken if a parameter value is NULL.

@variable

Is the variable that stores a parameter or a return parameter.

OUTPUT

Specifies that the module or command string returns a parameter. The matching parameter in the module or command string must also have been created by using the keyword OUTPUT (or OUT). Use this keyword when you use cursor variables as parameters.

If value is defined as OUTPUT of a module executed against a linked server, any changes to the corresponding @parameter performed by the OLE DB provider will be copied back to the variable at the end of the execution of module.

If OUTPUT parameters are being used and the intent is to use the return values in other statements within the calling batch or module, the value of the parameter must be passed as a variable, such as @parameter = @variable. You cannot execute a module by specifying OUTPUT for a parameter that is not defined as an OUTPUT parameter in the module. Constants cannot be passed to module by using OUTPUT; the return parameter requires a variable name. The data type of the variable must be declared and a value assigned before executing the procedure.

Return parameters can be of any data type except the LOB data types.

@string_variable

Is the name of a local variable. @string_variable can be any char, varchar, nchar, or nvarchar data type.

[N] 'tsql_string'

Is a constant string. tsql_string can be any nvarchar or varchar data type. If the N is included, the string is interpreted as nvarchar data type.

Remarks

For more information on using the EXECUTE statement, see EXECUTE (Transact-SQL).

To execute a stored procedure, list the parameter values in order, or name the parameter.

When using positional parameters the invocation of the procedure does not explicitly define the parameter name, but relies on the parameter value being in the expected parameter position. For example, EXEC [dbo].[sp_ExampleProc] 0, 1, 2 as positional parameters 0, 1, 2 being provided to the [dbo].[sp_ExampleProc] stored procedure.

When using named parameters the invocation explicitly identifies the parameter name while assigning its value as part of the stored procedure invocation. For example, EXEC [dbo].[sp_ExampleProc] @Param0 = 0, @Param1 = 1, @Param2 = 2 as named parameters being assigned the values 0, 1, 2, respectively, to the [dbo].[sp_ExampleProc] stored procedure.

When using both positional and named parameters, invocation respects the ordinal position of the parameters while having some combination of explicit naming of parameters and not. For example, EXEC [dbo].[sp_ExampleProc] 0, 1, @Param2 = 2 as a combination of ordinal and explicitly named parameters being provided for invocation of the [dbo].[sp_ExampleProc] stored procedure. Positional parameters can be used until first appearance of a named parameter. Once first named parameter is specified, all subsequent parameters also must be named.

Connections that use the .NET Framework Data Provider for SQL Server do not support positional parameters. This might affect connections from SSIS, SSRS, and SSAS, and linked servers.

Permissions

Any database user can use the EXECUTE statement. To execute a store procedure requires EXECUTE permission on the procedure and permission to perform the actions contained in the stored procedure. To execute a string requires permission to perform the actions contained in the string.

Examples

Example A: Basic Procedure Execution

Executing a stored procedure:

Copy image Copy Code
EXEC proc1;

Calling a stored procedure with name determined at runtime:

Copy image Copy Code
EXEC ('EXEC ' + @var);

Calling a stored procedure from within a stored procedure:

Copy image Copy Code
CREATE sp_first AS EXEC sp_second; EXEC sp_third;

Example B: Executing Strings

Executing a SQL string:

Copy image Copy Code
EXEC ('SELECT * FROM sys.types');

Executing a nested string:

Copy image Copy Code
EXEC ('EXEC (''SELECT * FROM sys.types'')');

Executing a string variable:

Copy image Copy Code
DECLARE @stringVar nvarchar(100);
SET @stringVar = N'SELECT name FROM' + ' sys.sql_logins';
EXEC (@stringVar);

Example C: Procedures with Parameters

The following example creates a procedure with parameters and demonstrates 3 ways to execute the procedure:

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
CREATE PROC ProcWithParameters
    @name nvarchar(50),
@color nvarchar (15)
AS
SELECT ProductKey, EnglishProductName, Color FROM [dbo].[DimProduct]
WHERE EnglishProductName LIKE @name
AND Color = @color;
GO

-- Executing using positional parameters
EXEC ProcWithParameters N'%arm%', N'Black';
-- Executing using named parameters in order
EXEC ProcWithParameters @name = N'%arm%', @color = N'Black';
-- Executing using named parameters out of order
EXEC ProcWithParameters @color = N'Black', @name = N'%arm%';
GO

See Also

Reference

CREATE PROCEDURE (SQL Server PDW)
ALTER PROCEDURE (SQL Server PDW)
DROP PROCEDURE (SQL Server PDW)


Expressions (SQL Server PDW)

[image]

Expressions (SQL Server PDW)
See Also Send Feedback

A combination of symbols and operators that SQL Server PDW evaluates to obtain a single data value. Simple expressions can be a single constant, variable, column, or scalar function. Operators can be used to join two or more simple expressions into a complex expression.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

Expression in a SELECT statement
<expression> ::=
{
    constant
    | scalar_function
    | column
    | variable
    | ( expression  )
    | { unary_operator } expression
    | expression { binary_operator } expression
}
[ COLLATE Windows_collation_name ]

Scalar Expression in a DECLARE, SET, IF…ELSE, or WHILE statement
<scalar_expression> ::=
{
    constant
    | scalar_function
    | variable
    | ( expression  )
    | (scalar_subquery )
    | { unary_operator } expression
    | expression { binary_operator } expression
}
[ COLLATE { Windows_collation_name ]

Arguments

constant

A symbol that represents a single, specific data value. For more information, see Constants.

scalar_function

A unit of SQL syntax that provides a specific service and returns a single value. Examples of scalar functions are SUM and CAST.

column

The name of a column in a table or view, fully qualified if necessary, and encased in the bracket delimiter '[…]' if the column name is a reserved word.

variable

Is the name of a variable.

expression

Any valid expression as defined in this topic. The parentheses are grouping operators that make sure that all the operators in the expression within the parentheses are evaluated before the resulting expression is combined with another.

scalar_subquery

A subquery that returns one value. For example, SELECT MAX(UnitPrice) FROM Products.

unary_operator

The - (Negative) unary operator.

Unary operators can be applied only to expressions that evaluate to any one of the numeric data types.

binary_operator

An operator that defines the way two expressions are combined to yield a single result. binary_operator can be an arithmetic operator, the assignment operator (=), a comparison operator, a logical operator, the string concatenation operator (+), or a unary operator. For more information about operators, see Operators (SQL Server PDW).

COLLATE Windows_collation_name

Specifies the collation for the expression. The collation must be one of the Windows collations supported by SQL Server. For a list of Windows collations supported by SQL Server, see Windows Collation Name (Transact-SQL). For examples using COLLATE, see COLLATE (Transact-SQL).

The default collation is the appliance-level collation Latin1_General_100_CI_AS_KS_WS. For more information about using collations in SQL Server PDW, see Collations (SQL Server PDW).

Expression Results

For a simple expression made up of a single constant, variable, scalar function, or column name: the data type, collation, precision, scale, and value of the expression is the data type, collation, precision, scale, and value of the referenced element.

When two expressions are combined by using comparison or logical operators, the resulting data type is Boolean and the value is one of the following: TRUE, FALSE, or UNKNOWN.

When two expressions are combined by using arithmetic, bitwise, or string operators, the operator determines the resulting data type. For more information, see Operators (SQL Server PDW).

Complex expressions made up of many symbols and operators evaluate to a single-valued result. The data type, collation, precision, and value of the resulting expression is determined by combining the component expressions, two at a time, until a final result is reached. The sequence in which the expressions are combined is defined by the precedence of the operators in the expression. For more information about operator precedence, see Operators (SQL Server PDW).

SQL Server PDW uses the SQL Server collation precedence rules to set the collation of each expression that evaluates to a character string. For more information, see Collation Precedence (Transact-SQL) on MSDN.

General Remarks

Implicit and Explicit Conversions

Two expressions can be combined by an operator if they both have data types supported by the operator and at least one of these conditions is true:

  • The expressions have the same data type.

  • The data type with the lower precedence can be implicitly converted to the data type with the higher data type precedence.

If the expressions do not meet these conditions, the CAST function can be used to explicitly convert the data type with the lower precedence to either the data type with the higher precedence or to an intermediate data type that can be implicitly converted to the data type with the higher precedence.

If there is no supported implicit or explicit conversion, the two expressions cannot be combined.

For more information about implicit data type conversions, see Data Types (SQL Server PDW). For more information about explicit data type conversions, see CAST and CONVERT (SQL Server PDW)

See Also


KILL (SQL Server PDW)

[image]

KILL (SQL Server PDW)
See Also Send Feedback

Terminates a SQL Server PDW user session based on the session ID. Use KILL, for example, to terminate a process that is blocking other important processes with locks, or to terminate a long-running query process that is using too many system resources.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

KILL 'session_id'
[;]

Arguments

session_id

The session ID of the SQL Server PDW process to terminate. session_id begins with the alphabetical characters 'SID'. These are case-sensitive and must be capitalized.

session_id is an nvarchar(32) value that is assigned to each user connection when the connection is made. It persists for the duration of the connection. When the connection ends, session_id is released.

Limitations and Restrictions

You cannot use KILL to kill your current session.

Metadata

Examples of system views that return PDW session IDs:

  • sys.dm_pdw_exec_sessions (SQL Server PDW),

    sys.dm_pdw_exec_requests (SQL Server PDW)

  • sys.dm_pdw_query_stats_xe (SQL Server PDW)

  • sys.dm_pdw_query_stats_xe_file (SQL Server PDW)

  • sys.dm_pdw_resource_waits (SQL Server PDW)

  • sys.dm_pdw_waits (SQL Server PDW)

  • sys.dm_pdw_lock_waits (SQL Server PDW)

Permissions

Requires the ALTER ANY CONNECTION permission.

General Remarks

Use KILL very carefully, especially when critical processes are running.

Examples

The following example shows how to terminate session ID SID535.

Copy image Copy Code
KILL 'SID535';

See Also

Reference

SESSION_ID (SQL Server PDW)


IDENTITY Property Workaround (SQL Server PDW)

[image]

IDENTITY Property Workaround (SQL Server PDW)
See Also Send Feedback

SQL Server PDW does not support the IDENTITY property for a column of a table or the sequence number object. The following example demonstrates a workaround that might be useful, depending upon your needs.

Remarks

Data entering a large data warehouse is usually fully structured and validated. If a table has an IDENTITY column, the values have typically been assigned when the data was first created. The IDENTITY value in the table is inherently preserved during the transfer to the data warehouse, the same way any other value is preserved. If an IDENTITY value is not present but is needed, either create the source table with an IDENTITY column before transferring the table to the data warehouse, or use the following workaround to add a numeric counter column to the data as it is transferred. This method uses an int data type, but a bigint could be used. This example simulates a seed of 1 and increment of 1.

Note Note

This workaround does not create a true IDENTITY value. The column value is not enforced as unique or as an identity column. The column does not have the IDENTITY property.

Examples

The following example demonstrates a workaround to populate a table with a column that simulates an identity column. The example creates and populates a replicated staging table (Basic_Stage) to hold the data that will be transferred. Then the example creates the destination table (Prod_Identity_Test). Then the data is copied from the staging to the destination table, using the ROW_NUMBER function to add sequential number to the IDCol column.

Note Note

The staging table must be replicated to work properly for this example. The destination table can be replicated or distributed.

Copy image Copy Code
-- Create and load a sample staging table
CREATE TABLE Basic_Stage
(IncomingData int NOT NULL)
WITH (DISTRIBUTION = REPLICATE);

INSERT INTO basic_stage (IncomingData) Values (1);
INSERT INTO basic_stage (IncomingData) Values (2);
INSERT INTO basic_stage (IncomingData) Values (3);
INSERT INTO basic_stage (IncomingData) Values (4);
INSERT INTO basic_stage (IncomingData) Values (5);
SELECT * FROM basic_stage;

--Create a sample production table
Create Table Prod_Identity_Test
(IDCol INT NOT NULL,
OtherCols Varchar(10))
WITH (DISTRIBUTION=HASH(IDCol));

-- Populate the Prod_Identity_Test table with data
-- from the Basic_Stage table.
INSERT INTO Prod_Identity_Test
SELECT (ROW_NUMBER() OVER ( ORDER BY p.IncomingData)) + maxID.maxID,
p.IncomingData
FROM
    (SELECT IncomingData  FROM  basic_stage) AS p
CROSS JOIN
    (Select isnull((max(IDCol)),0) AS maxID
     FROM Prod_Identity_Test)
     AS maxID;
-- The isnull ensures the first value is 1 when no records exist

SELECT * FROM Prod_Identity_Test ORDER BY IDCol;

See Also

Reference

ROW_NUMBER (SQL Server PDW)


PRINT (SQL Server PDW)

[image]

PRINT (SQL Server PDW)
See Also Send Feedback

Returns a user-defined message to the client.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

PRINT msg_str | @local_variable | string_expr

Arguments

msg_str

Is a character string or Unicode string constant.

@local_variable

Is a variable of any valid character data type. @local_variable must be char, nchar, varchar, or nvarchar, or it must be able to be implicitly converted to those data types.

string_expr

Is an expression that returns a string. Can include concatenated literal values, functions, and variables.

Remarks

A message string can be up to 8,000 characters long if it is a non-Unicode string, and 4,000 characters long if it is a Unicode string. Longer strings are truncated.

RAISERROR can also be used to return messages. The principal difference between the PRINT and RAISERROR is that PRINT supports expressions as the parameter and RAISERROR only supports literals.

Examples

A. Conditionally executing print

The following example uses the PRINT statement to conditionally return a message.

Copy image Copy Code
IF DB_ID() = 1
    PRINT N'The current database is master.';
ELSE
    PRINT N'The current database is not master.';
GO

B. Building and displaying a string

The following example converts the results of the GETDATE function to a nvarchar data type and concatenates it with literal text to be returned by PRINT.

Copy image Copy Code
-- Build the message text by concatenating
-- strings and expressions.
PRINT N'This message was printed on '
    + RTRIM(CAST(GETDATE() AS nvarchar(30)))
    + N'.';
GO
-- This example shows building the message text
-- in a variable and then passing it to PRINT.
DECLARE @PrintMessage nvarchar(50);
SET @PrintMessage = N'This message was printed on '
    + RTRIM(CAST(GETDATE() AS nvarchar(30)))
    + N'.';
PRINT @PrintMessage;
GO

See Also

Reference

RAISERROR (SQL Server PDW)
TRY...CATCH (SQL Server PDW)
ERROR_MESSAGE (SQL Server PDW)
ERROR_NUMBER (SQL Server PDW)
ERROR_PROCEDURE (SQL Server PDW)
ERROR_SEVERITY (SQL Server PDW)
ERROR_STATE (SQL Server PDW)
THROW (SQL Server PDW)


USE (SQL Server PDW)

[image]

USE (SQL Server PDW)
See Also Send Feedback

Specifies the database context in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

USE database_name
[;]

Arguments

database_name

The name of the database context to use.

Permissions

Requires CONNECT permission in the target database.

General Remarks

When a login connects to SQL Server PDW, it acquires the database context of its default database.

Locking

Takes a shared lock on the database object.

Examples

The following example changes the database context to the AccountingDB database.

Copy image Copy Code
USE AccountingDB;

See Also


Operators (SQL Server PDW)

[image]

Operators (SQL Server PDW)
See Also Send Feedback

Describes the operators and operator precedence rules used in SQL Server PDW SQL expressions.

Table of Contents

  • Arithmetic Operators

  • Comparison Operators

  • Comparison Operators

  • Comparison Operators

  • Operator Precedence

Arithmetic Operators

Arithmetic operators perform mathematical operations on two expressions of one or more of the data types of the numeric data type category. For more information about data type categories and rounding and truncation behaviors when performing mathematical operations on numeric data types, see Data Types (SQL Server PDW).

Operator | Meaning
+ (Add) | Addition
- (Subtract) | Subtraction
* (Multiply) | Multiplication
/ (Divide) | Division
% (Modulo) | Returns the integer remainder of a division. For example, 12 % 5 = 2 because the remainder of 12 divided by 5 is 2. (For use of % as a wildcard character, see Wildcard Character(s) to Match (SQL Server PDW).)

The plus (+) and minus (-) operators can also be used to perform arithmetic operations on datetime values.

For more information about the precision and scale of the result of an arithmetic operation, see "Precision, Scale, and Length for Data Types" in the Data Types (SQL Server PDW) topic.

Understanding Rounding and Truncation in Mathematical Operations

  • Calculations performed on numeric data types may result in either rounding or truncation.

This section describes the logic applied to the handling of int, decimal, and float values in mathematical calculations:

  • SQL Server PDW truncates (removes the decimal) during integer division, rather than rounding.

  • SQL Server PDW uses the "round odd down" method when determining rounding on values ending with "5". This means that the digit preceding the "5" determines whether the value will be rounded up or down. If the digit to the left of the "5" is even, the value will be rounded up. If the digit to the left of the "5" is odd, the value will be rounded down. This method is used so that results are not biased up when the digit determining the rounding is "5".

Rounding and Truncation Examples

The following table provides examples of rounding and truncation behavior.

Formula | Result | Explanation
5/4 7/4 3/2 5/2 7/2 9/2 | 1 1 1 2 3 4 | These are integer evaluations, so the results are truncated to an integer.
5*1.000/4 7.000/4 3.000/2 5.000/2 7*1.000/2 9*1.00/2 | 1.250000... 1.750000... 1.500000... 2.500000... 3.500000... 4.500000... | These are decimal evaluations, so the decimal precision is preserved.
ROUND(5*1.000/4) ROUND(7.000/4) ROUND(3.000/2,0) ROUND(5.000/2,0) ROUND(7*1.000/2,0) ROUND(9*1.000/2,0) | 1.0 2.0 1.0 3.0 3.0 5.0 | Use of the ROUND function (see ROUND (SQL Server PDW)) causes the decimal to be rounded up or down. Since the resulting type is a decimal type, the decimal place is preserved in the result.

Comparison Operators

Comparison operators test whether two expressions are the same. Comparison operators can be used on all expression. The following table lists the SQL comparison operators.

Operator | Meaning
= (Equals) | Equal to
> (Greater Than) | Greater than
< (Less Than) | Less than
>= (Greater Than or Equal To) | Greater than or equal to
<= (Less Than or Equal To) | Less than or equal to
<> (Not Equal To) | Not equal to
!= (Not Equal To) | Not equal to (not ISO standard)

Boolean Data Types with Comparison Operators

The result of a comparison operator has the Boolean data type. This has three values: TRUE, FALSE, and UNKNOWN. Expressions that return a Boolean data type are known as Boolean expressions.

Unlike other SQL Server PDW data types, a Boolean data type cannot be specified as the data type of a table column or variable, and cannot be returned in a result set.

An operator that has one or two NULL expressions returns UNKNOWN.

Expressions with Boolean data types are used in the WHERE clause to filter the rows that qualify for the search conditions.

Copy image Copy Code
SELECT id, firstName, lastName FROM Customer WHERE orderTotal > 100;

Logical Operators

Logical operators test for the truth of some condition. Logical operators, like comparison operators, return a Boolean data type with a value of TRUE, FALSE, or UNKNOWN.

Operator | Meaning
AND | TRUE if both Boolean expressions are TRUE.
BETWEEN | TRUE if the operand is within a range.
EXISTS | TRUE if a subquery contains any rows.
IN | TRUE if the operand is equal to one of a list of expressions.
IS [NOT] NULL | TRUE if the following expression is null.
LIKE | TRUE if the operand matches a pattern.
NOT | Reverses the value of any other Boolean operator.
OR | TRUE if either Boolean expression is TRUE.

String Concatenation Operator

The plus sign (+) is the string concatenation operator. All other string manipulation is handled by using string functions such as SUBSTRING.

In concatenating data of the varchar, or char data types, the empty string is interpreted as an empty string. For example, 'abc' + '' + 'def' is stored as 'abcdef'.

Examples

The following is an example of concatenation.

Copy image Copy Code
SELECT ('abc' + 'def') FROM DimEmployee;

Here is the result set.

Copy image Copy Code
------
abcdef

The following query displays names of the first four contacts under the Moniker column in last name, first name order, with a comma and space after the last name.

Copy image Copy Code
SELECT LastName + ', ' + FirstName AS Moniker
FROM Person.Contact
WHERE ContactID < 5;

Here is the result set.

Copy image Copy Code
Moniker
-------------------------
Achong, Gustavo
Abel, Catherine
Abercrombie, Kim
Acevedo, Humberto

Other data types, such as datetime and date, must be converted to character strings by using the CAST conversion function before they can be concatenated with a string.

Copy image Copy Code
SELECT 'The due date is ' + CAST(DueDate AS varchar(128))
FROM Sales.SalesOrderHeader
WHERE SalesOrderID = 43659;

Here is the result set.

Copy image Copy Code
---------------------------------------
The due date is Jul 13 2001 12:00AM

When the input strings both have the same collation, the output string has the same collation as the inputs. When the input strings have different collations, the rules of collation precedence determine the collation of the output string. See Collation Precedence in the Data Types (SQL Server PDW) topic for more information.

Operator Precedence

When a complex expression has multiple operators, operator precedence determines the sequence in which the operations are performed. The order of execution can significantly affect the resulting value.

Operators have the precedence levels shown in the following table. An operator on higher levels is evaluated before an operator on a lower level.

Level | Operators
1 | * (Multiply), / (Division), MOD
2 | + (Positive), - (Negative), + (Add), (+ Concatenate / || (Concatenate)), - (Subtract)
3 | =, >, <, >=, <=, <>, != (Comparison operators)
4 | NOT
5 | AND
6 | ALL, BETWEEN, IN, LIKE, OR

When two operators in an expression have the same operator precedence level, they are evaluated left to right based on their position in the expression. For example, in the following expression, the subtraction operator is evaluated before the addition operator.

Copy image Copy Code
SELECT id, first_name, last_name FROM Customer WHERE (RecentSales - LastYearSales + Commission) > 1000;

Use parentheses to override the defined precedence of the operators in an expression. Everything within the parentheses is evaluated first to yield a single value before that value can be used by any operator outside the parentheses.

For example, in the following expression, the multiplication operator has a higher precedence than the addition operator. Therefore, it is evaluated first.

Copy image Copy Code
SELECT id, first_name, last_name FROM Customer WHERE (RecentSales - Commission * 10) > 1000;

In the following expression, the parentheses cause the addition to be performed first. The expression result is 18.

Copy image Copy Code
SELECT id, first_name, last_name FROM Customer WHERE ((RecentSales - Commission) * 10) > 1000;

If an expression has nested parentheses, the most deeply nested expression is evaluated first. The following example contains nested parentheses, with the expression 5 - 3 in the most deeply nested set of parentheses. This expression yields a value of 2. Then, the addition operator (+) adds this result to 4. This yields a value of 6. Finally, the 6 is multiplied by 2 to yield an expression result of 12.

Copy image Copy Code
SELECT id, first_name, last_name FROM Customer WHERE 2 * (4 + (5 - 3) ) > 100;
-- Evaluates to 2 * (4 + 2) which then evaluates to 2 * 6, and
-- yields an expression result of 12.

See Also

Reference

Expressions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


+ (Add) (SQL Server PDW)

[image]

+ (Add) (SQL Server PDW)
See Also Send Feedback

Adds two numbers (an arithmetic addition operator).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression + expression

Arguments

expression

Any valid expression of any one of the data types in the numeric category.

Result Types

Returns the data type of the argument with the higher precedence. For more information, see Data Type Precedence in the Data Types (SQL Server PDW) topic.

Examples

Using the addition operator to calculate the total number of hours away from work for each employee

The following example finds the total number of hours away from work for each employee by adding the number of hours taken for vacation and the number of hours taken as sick leave and sorts the results in ascending order.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, VacationHours, SickLeaveHours,
    VacationHours + SickLeaveHours AS TotalHoursAway
FROM DimEmployee
ORDER BY TotalHoursAway ASC;

See Also

Reference

Operators (SQL Server PDW)
CAST and CONVERT (SQL Server PDW)
Functions (SQL Server PDW)
SELECT (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


+ (String Concatenation) (SQL Server PDW)

[image]

+ (String Concatenation) (SQL Server PDW)
See Also Send Feedback

An operator in a string expression that concatenates two or more character or binary strings, columns, or a combination of strings and column names into one expression (a string operator).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression + expression

Arguments

expression

Is any valid expression of any one of the data types in the character and binary data type category. Both expressions must be of the same data type, or one expression must be implicitly convertible to the data type of the other expression.

An explicit conversion to character data must be used when concatenating binary strings and any characters between the binary strings.

Result Types

Returns the data type of the argument with the highest precedence. For more information, see Data Type Precedence in Data Types (SQL Server PDW).

When two char or varchar expressions are concatenated, the length of the resulting expression is the sum of the lengths of the two source expressions or 8,000 characters, whichever is less.

When two nchar or nvarchar expressions are concatenated, the length of the resulting expression is the sum of the lengths of the two source expressions or 4,000 characters, whichever is less.

General Remarks

The string concatenation operator behaves differently when it works with an empty, zero-length string than when it works with null or unknown values.

A zero-length character string can be specified as two single quotation marks without any characters inside the quotation marks. A zero-length binary string can be specified as 0x without any byte values specified in the hexadecimal constant. Concatenating a zero-length string always concatenates the two specified strings.

The result of the concatenation of a string and a null, by contrast, is a null result.

If the result of the concatenation of strings exceeds the limit of 8,000 bytes, the result is truncated. However, if at least one of the strings concatenated is a large value type, truncation does not occur.

Examples

A. Using string concatenation

The following example creates a single column under the column heading Name from multiple-character columns, with the last name of the contact followed by a comma, a single space, and then the first name of the contact. The result set is in ascending, alphabetical order by the last name, and then by the first name.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT (LastName + ', ' + FirstName) AS Name
FROM DimEmployee
ORDER BY LastName ASC, FirstName ASC;

B. Using multiple string concatenation

The following example concatenates multiple strings to form one long string to display the last name and the first initial of the vice presidents within a sample database. A comma is added after the last name and a period after the first initial.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT (LastName + ', ' + SUBSTRING(FirstName, 1, 1) + '.') AS Name, Title
FROM DimEmployee
WHERE Title LIKE '%Vice Pres%'
ORDER BY LastName ASC;

Here is the result set.

Copy image Copy Code
Name               Title
-------------      ---------------
Duffy, T.          Vice President of Engineering
Hamilton, J.       Vice President of Production
Welcker, B.        Vice President of Sales

See Also

Reference

CAST and CONVERT (SQL Server PDW)
Expressions (SQL Server PDW)
Functions (SQL Server PDW)
Operators (SQL Server PDW)
SELECT (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


- (Negative) (SQL Server PDW)

[image]

- (Negative) (SQL Server PDW)
See Also Send Feedback

Reverses the sign of a numeric expression or value (a unary operator).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

- numeric_expression

Arguments

numeric_expression

Any valid expression of any one of the numeric data types.

Result Types

Returns the data type of numeric_expression, except that an unsigned tinyint expression is promoted to a signed smallint result.

Examples

A. Returning the negative of a positive constant

The following example returns the negative of a positive constant.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP (1) - 17 FROM DimEmployee;

Returns

Copy image Copy Code
-17

B. Returning the positive of a negative constant

The following example returns the positive of a negative constant.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP (1) – ( - 17) FROM DimEmployee;

Returns

Copy image Copy Code
17

C. Returning the negative of a column

The following example returns the negative of the BaseRate value for each employee in the dimEmployee table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT - BaseRate FROM DimEmployee;

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


- (Subtract) (SQL Server PDW)

[image]

- (Subtract) (SQL Server PDW)
See Also Send Feedback

Subtracts two numbers (an arithmetic subtraction operator).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression - expression

Arguments

expression

Any valid expression of any one of the data types of the numeric data type category.

Result Types

Returns the data type of the argument with the higher precedence. For more information, see Data Type Precedence in the Data Types (SQL Server PDW) topic.

Examples

The following example calculates the difference in a base rate between the employee with the highest base rate and the employee with the lowest tax rate, from the dimEmployee table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT MAX(BaseRate) - MIN(BaseRate) AS BaseRateDifference
FROM DimEmployee;

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
SELECT (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


* (Multiply) (SQL Server PDW)

[image]

* (Multiply) (SQL Server PDW)
See Also Send Feedback

Multiplies two expressions (an arithmetic multiplication operator).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression * expression

Arguments

expression

Any valid expression of any one of the data types of the numeric data type category.

Result Types

Returns the data type of the argument with the higher precedence. For more information, see Data Type Precedence in Data Types (SQL Server PDW).

Examples

The following example retrieves the first and last name of employees in the dimEmployee table, and calculates the pay for VacationHours for each..

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, BaseRate * VacationHours AS VacationPay
FROM DimEmployee
ORDER BY lastName ASC;

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
Operators (SQL Server PDW)
SELECT (SQL Server PDW)
WHERE (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


/ (Divide) (SQL Server PDW)

[image]

/ (Divide) (SQL Server PDW)
See Also Send Feedback

Divides one number by another (an arithmetic division operator).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

dividend / divisor

Arguments

dividend

Any valid expression of any one of the data types of the numeric data type category.

divisor

Any valid expression of any one of the data types of the numeric data type category.

Result Types

Returns the data type of the argument with the higher precedence. For more information, see Data Type Precedence in Data Types (SQL Server PDW).

Remarks

The value returned by the / operator is the quotient of the first expression divided by the second expression.

If an integer dividend is divided by an integer divisor, the result is an integer that has any fractional part of the result truncated.

Examples

The following example uses the division arithmetic operator to calculate a simple ratio of each employees’ vacation hours to sick hours.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, VacationHours/SickLeaveHours AS PersonalTimeRatio
FROM DimEmployee;

See Also

Reference

* (Multiply) (SQL Server PDW)
Functions (SQL Server PDW)
Operators (SQL Server PDW)
SELECT (SQL Server PDW)
WHERE (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


Modulo (SQL Server PDW)

[image]

Modulo (SQL Server PDW)
See Also Send Feedback

Uses the % operator to returns the integer remainder of a division. For example, 12 % 5 = 2 because the remainder of 12 divided by 5 is 2.

Note Note

For information about using % as a wildcard character, see Wildcard Character(s) to Match (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

numerator  %  denominator

Arguments

numerator

An expression or constant of the numerator. Can be an exact numeric or approximate numeric data type category.

denominator

An expression or constant of the denominator. Can be an exact numeric or approximate numeric data type category.

Return Types

int

Examples

The following example shows results for the % operator when dividing 3 by 2.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1) 3%2 FROM dimEmployee;

Here is the result set.

Copy image Copy Code
---------
1

See Also

Reference

/ (Divide) (SQL Server PDW)
Expressions (SQL Server PDW)
Constants (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


Wildcard Character(s) to Match (SQL Server PDW)

[image]

Wildcard Character(s) to Match (SQL Server PDW)
See Also Send Feedback

The % character matches any string of zero or more characters. The % character can be used as either a prefix or a suffix.

Note Note

For information about using % as the arithmetic modulo operator, see Modulo (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Examples

The following example returns all the names of people in the dimEmployee table where the first name starts with D.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName
FROM dimEmployee
WHERE FirstName LIKE 'D%';

See Also

Reference

LIKE (SQL Server PDW)


= (Equals) (SQL Server PDW)

[image]

= (Equals) (SQL Server PDW)
See Also Send Feedback

Compares the equality of two expressions (a comparison operator).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression = expression

Arguments

expression

Any valid expression. If the expressions are not of the same data type, the data type for one expression must be implicitly convertible to the data type of the other. The conversion is based on the rules of data type precedence. For more information, see Data Type Precedence in Data Types (SQL Server PDW).

Result Types

Boolean

Remarks

When you compare two NULL expressions, the result is NULL.

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


> (Greater Than) (SQL Server PDW)

[image]

> (Greater Than) (SQL Server PDW)
See Also Send Feedback

Compares two expressions (a comparison operator). When you compare non-null expressions, the result is TRUE if the left operand has a higher value than the right operand; otherwise, the result is FALSE. If either or both operands are NULL, the result is UNKNOWN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression > expression

Arguments

expression

Any valid expression. Both expressions must have implicitly convertible data types. The conversion depends on the rules of data type precedence. See Data Type Precedence in the Data Types (SQL Server PDW) topic.

Result Types

Boolean

Remarks

Comparisons involving null values, such as (NULL > 100) or (100 > NULL) return UNKNOWN; a comparison that results in UNKNOWN is not returned with either TRUE or FALSE results.

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


[image]

< (Less Than) (SQL Server PDW)
See Also Send Feedback

Compares two expressions (a comparison operator). When you compare non-null expressions, the result is TRUE if the left operand has a value lower than the right operand; otherwise, the result is FALSE. If either or both operands are NULL, the result is UNKNOWN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression < expression

Arguments

expression

Any valid expression. Both expressions must have implicitly convertible data types. The conversion depends on the rules of data type precedence. For more information, see Data Type Precedence in Data Types (SQL Server PDW).

Result Types

Boolean

Remarks

Comparisons involving null values, such as (NULL < 100) or (100 < NULL) return UNKNOWN; a comparison that results in UNKNOWN is not returned with either TRUE or FALSE results.

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
> (Greater Than) (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


>= (Greater Than or Equal To) (SQL Server PDW)

[image]

>= (Greater Than or Equal To) (SQL Server PDW)
See Also Send Feedback

Compares two expressions (a comparison operator). When you compare non-null expressions, the result is TRUE if the right operand has a value lower than or equal to the left operand; otherwise, the result is FALSE.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression > = expression

Arguments

expression

Is any valid expression. Both expressions must have implicitly convertible data types. The conversion depends on the rules of data type precedence. For more information, see Data Type Precedence in Data Types (SQL Server PDW).

Result Types

Boolean

Remarks

When you compare non-null expressions, the result is TRUE if the left operand has a value greater than or equal to that of the right operand; otherwise, the result is FALSE.

Comparisons involving null values, such as (NULL >= 100) or (100 >= NULL) return UNKNOWN; a comparison that results in UNKNOWN is not returned with either TRUE or FALSE results.

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
= (Equals) (SQL Server PDW)
> (Greater Than) (SQL Server PDW)
<= (Less Than or Equal To) (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


[image]

<= (Less Than or Equal To) (SQL Server PDW)
See Also Send Feedback

Compares two expressions (a comparison operator). When you compare non-null expressions, the result is TRUE if the left operand has a value lower than or equal to the right operand; otherwise, the result is FALSE.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression <= expression

Arguments

expression

Any valid expression. Both expressions must have implicitly convertible data types. The conversion depends on the rules of data type precedence. For more information, see Data Type Precedence in Data Types (SQL Server PDW).

Result Types

Boolean

Remarks

Comparisons involving null values, such as (NULL <= 100) or (100 <= NULL) return UNKNOWN; a comparison that results in UNKNOWN is not returned with either TRUE or FALSE results.

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
= (Equals) (SQL Server PDW)
< (Less Than) (SQL Server PDW)
>= (Greater Than or Equal To) (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


[image]

<> (Not Equal To) (SQL Server PDW)
See Also Send Feedback

Compares two expressions (a comparison operator). The result is TRUE if the left non-null operand is not equal to the right non-null operand; otherwise; the result is FALSE. If either or both operands are null, the result is UNKNOWN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression < > expression

Arguments

expression

Is any valid expression. Both expressions must have implicitly convertible data types. The conversion depends on the rules of data type precedence. For more information, see Data Type Precedence in Data Types (SQL Server PDW).

Result Types

Boolean

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


!= (Not Equal To) (SQL Server PDW)

[image]

!= (Not Equal To) (SQL Server PDW)
See Also Send Feedback

Tests whether one expression is not equal to another expression (a comparison operator). If either or both operands are NULL, NULL is returned. It functions the same as the <> (Not Equal To) comparison operator.

Topic link icon Syntax Conventions (SQL Server PDW)

See Also


AND (SQL Server PDW)

[image]

AND (SQL Server PDW)
See Also Send Feedback

Combines two Boolean expressions and returns TRUE when both expressions are true. When more than one logical operator is used in a statement, the AND operators are evaluated first. You can change the order of evaluation by using parentheses.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

boolean_expression AND boolean_expression

Arguments

boolean_expression

Is any valid expression that returns a Boolean value: TRUE, FALSE, or UNKNOWN.

Result Types

Boolean

Result Value

Returns TRUE when both expressions are true.

Remarks

The following chart shows the outcomes when you compare TRUE and FALSE values by using the AND operator.

Expression1 | Expression2 | Result
TRUE | TRUE | TRUE
FALSE | FALSE | FALSE
TRUE | FALSE | FALSE
FALSE | TRUE | FALSE
UNKNOWN | [TRUE | FALSE] | UNKNOWN
[TRUE | FALSE] | UNKNOWN | UNKNOWN

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
Operators (SQL Server PDW)
SELECT (SQL Server PDW)
WHERE (SQL Server PDW)


BETWEEN (SQL Server PDW)

[image]

BETWEEN (SQL Server PDW)
See Also Send Feedback

Specifies a range to test.

BETWEEN returns TRUE if the value of test_expression is greater than or equal to the value of begin_expression and less than or equal to the value of end_expression.

NOT BETWEEN returns TRUE if the value of test_expression is less than the value of begin_expression or greater than the value of end_expression.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

test_expression [ NOT ] BETWEEN begin_expression AND end_expression

Arguments

test_expression

The expression to test for in the range defined by begin_expressionand end_expression. test_expression must be the same data type as both begin_expression and end_expression.

NOT

Negates the result of the predicate.

begin_expression

Any valid expression. begin_expression must be the same data type as both test_expression and end_expression.

end_expression

Any valid expression. end_expression must be the same data type as both test_expressionand begin_expression.

AND

Acts as a placeholder that indicates test_expression should be within the range indicated by begin_expression and end_expression.

Result Types

Boolean

Remarks

To specify an exclusive range, use the greater-than (>) and less-than operators (<). If any input to the BETWEEN or NOT BETWEEN predicate is NULL, the result is UNKNOWN.

Examples

A. Using BETWEEN

The following example returns the employees of a company that have an hourly pay rate between 27 and 30, inclusive.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, BaseRate
FROM dimEmployee
WHERE BaseRate BETWEEN 27 AND 30
ORDER BY BaseRate DESC;

B. Using >= and <= instead of BETWEEN

The following example uses the greater than or equal to (>=) and less than or equal to (<=) operators to perform the same query as the BETWEEN query above.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, BaseRate
FROM dimEmployee
WHERE BaseRate >= 27 AND BaseRate <= 30
ORDER BY BaseRate DESC;

C. Using NOT BETWEEN

The following example finds all rows outside the range of 27 through 30.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, BaseRate
FROM dimEmployee
WHERE BaseRate NOT BETWEEN 27 AND 30
ORDER BY BaseRate DESC;

D. Using BETWEEN with date values

The following example retrieves employees with BirthDate values are between '1950-01-01' and '1969-12-31', inclusive.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, BirthDate
FROM dimEmployee
WHERE BirthDate BETWEEN '1950-01-01' AND '1969-12-31'
ORDER BY BirthDate ASC;

See Also

Reference

> (Greater Than) (SQL Server PDW)
< (Less Than) (SQL Server PDW)
Expressions (SQL Server PDW)
Functions (SQL Server PDW)
Operators (SQL Server PDW)
SELECT (SQL Server PDW)
WHERE (SQL Server PDW)


EXISTS (SQL Server PDW)

[image]

EXISTS (SQL Server PDW)
See Also Send Feedback

Specifies a subquery to test for the existence of rows.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

EXISTS ( subquery )

Arguments

subquery

A restricted SELECT statement. For more information, see Subqueries (SQL Server PDW).

Result Types

Boolean

Result Values

Returns TRUE if a subquery contains any rows.

Examples

A. Using EXISTS

The following example identifies whether any rows in the ProspectiveBuyer table could be matches to rows in the DimCustomer table. The query will return rows only when both the LastName and BirthDate values in the two tables match.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT a.LastName, a.BirthDate
FROM DimCustomer AS a
WHERE EXISTS
(SELECT *
    FROM dbo.ProspectiveBuyer AS b
    WHERE (a.LastName = b.LastName) AND (a.BirthDate = b.BirthDate));

B. Using NOT EXISTS

NOT EXISTS works as the opposite as EXISTS. The WHERE clause in NOT EXISTS is satisfied if no rows are returned by the subquery. The following example finds rows in the DimCustomer table where the LastName and BirthDate do not match any entries in the ProspectiveBuyers table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT a.LastName, a.BirthDate
FROM DimCustomer AS a
WHERE NOT EXISTS
(SELECT *
    FROM dbo.ProspectiveBuyer AS b
    WHERE (a.LastName = b.LastName) AND (a.BirthDate = b.BirthDate));

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
WHERE (SQL Server PDW)


IN (SQL Server PDW)

[image]

IN (SQL Server PDW)
See Also Send Feedback

Determines whether a specified value matches any value in a subquery or a list.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

test_expression [ NOT ] IN
    ( subquery | expression [ ,...n ]
    )

Arguments

test_expression

Any valid expression.

subquery

A subquery that has a result set of one column. This column must have the same data type as test_expression.

expression [ ,... n ]

A list of expressions to test for a match. All expressions must be of the same type as test_expression.

Result Types

Boolean

Result Value

If the value of test_expression is equal to any value returned by subquery or is equal to any expression from the comma-separated list, the result value is TRUE; otherwise, the result value is FALSE.

Using NOT IN negates the subquery value or expression.

Caution note Caution

Any null values returned by subquery or expression that are compared to test_expression using IN or NOT IN return UNKNOWN. Using null values together with IN or NOT IN can produce unexpected results.

Examples

A. Using IN and NOT IN

The following example finds all entries in the FactInternetSales table that match SalesReasonKey values in the DimSalesReason table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT * FROM FactInternetSalesReason
WHERE SalesReasonKey
IN (SELECT SalesReasonKey FROM DimSalesReason);

The following example finds all entries in the FactInternetSalesReason table that do not match SalesReasonKey values in the DimSalesReason table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT * FROM FactInternetSalesReason
WHERE SalesReasonKey
NOT IN (SELECT SalesReasonKey FROM DimSalesReason);

B. Using IN with an expression list

The following example finds all IDs for the salespeople in the DimEmployee table for employees who have a first name that is either Mike or Michael.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName
FROM DimEmployee
WHERE FirstName IN ('Mike', 'Michael');

See Also

Reference

Expressions (SQL Server PDW)
CASE (SQL Server PDW)
Functions (SQL Server PDW)
Operators (SQL Server PDW)
SELECT (SQL Server PDW)
WHERE (SQL Server PDW)


IS [NOT] NULL (SQL Server PDW)

[image]

IS [NOT] NULL (SQL Server PDW)
See Also Send Feedback

Determines whether a specified expression is null.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression IS [ NOT ] NULL

Arguments

expression

Any valid expression.

NOT

Negates the Boolean result. The predicate reverses its return values, returning TRUE if the value is not null, and FALSE if the value is NULL.

Result Types

Boolean

Return Values

If the value of expression is null, IS NULL returns TRUE; otherwise, it returns FALSE.

If the value of expression is null, IS NOT NULL returns FALSE; otherwise, it returns TRUE.

To replace NULL values with a specified value, use ISNULL.

Remarks

To determine whether an expression is null, use IS NULL or IS NOT NULL instead of comparison operators (such as = or <>). Comparison operators return UNKNOWN when either or both arguments are null.

Examples

The following example returns the full names of all employees with middle initials.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, MiddleName
FROM DIMEmployee
WHERE MiddleName IS NOT NULL
ORDER BY LastName DESC;

See Also

Reference

CASE (SQL Server PDW)
NULL and UNKNOWN (SQL Server PDW)
Expressions (SQL Server PDW)
LIKE (SQL Server PDW)
Operators (SQL Server PDW)
WHERE (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


LIKE (SQL Server PDW)

[image]

LIKE (SQL Server PDW)
See Also Send Feedback

Determines whether a specific character string matches a specified pattern. A pattern can include regular characters and wildcard characters.

This topic includes discussions of both the LIKE operator and Pattern Matching in Search Conditions.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

match_expression [ NOT ] LIKE pattern

Arguments

match_expression

Any valid expression of character data type.

pattern

The specific string of characters to search for in match_expression. pattern can include valid wildcard characters. pattern can be a maximum of 8,000 bytes. For more information about pattern values, see Pattern Matching in Search Conditions below.

Result Types

Boolean

Result Value

LIKE returns TRUE if match_expression matches the specified pattern.

Remarks

If any one of the arguments is not a character string data type, SQL Server PDW converts it to character string data type, if it is possible.

Pattern Matching in Search Conditions

Pattern matching in search conditions is used with either the LIKE keyword or the PATINDEX function, and uses a regular expression to contain the pattern that the values are matched against. The pattern contains the character string to search for, which can contain any combination of regular characters and four wildcards.

During pattern matching, regular characters must exactly match the characters specified in the character string. However, wildcard characters can be matched with arbitrary fragments of the character string.

Wildcard | Meaning
% | Any string of zero or more characters. For example, WHERE title LIKE '%computer%' finds all book titles with the word 'computer' anywhere in the book title.
_ | Any single character. For example, WHERE au_fname LIKE '_ean' finds all four-letter first names that end with 'ean' ("Dean", "Sean", and so on).
[ ] | Any single character within the specified range ([a-f]) or set ([abcdef]). For example, WHERE au_lname LIKE '[C-P]arsen' finds author last names ending with arsen and starting with any single character between C and P, inclusive. For example Carsen, Larsen, and Karsen are matching values. In range searches, the characters included in the range may vary depending on the sorting rules of the collation.
[^] | Any single character not within the specified range ([^a-f]) or set ([^abcdef]). For example, WHERE au_lname LIKE 'de[^l]%' finds all author last names starting with de and where the following letter is not l.

Enclose the wildcards and the character string in single quotation marks, for example:

  • LIKE 'Mc%' searches for all strings that begin with the letters Mc (For example, 'McBadden').

  • LIKE '%inger' searches for all strings that end with the letters inger, such as Ringer, Stringer.

  • LIKE '%en%' searches for all strings that contain the letters en anywhere in the string such as Bennet, Green, McBadden.

  • LIKE '_heryl' searches for all six-letter names ending with the letters heryl such as Cheryl, Sheryl.

  • LIKE '[CK]ars[eo]n' searches for values such as Carsen, Karsen, Carson, and Karson.

  • LIKE '[M-Z]inger' searches for all names ending with the letters inger that begin with any single letter from M through Z, such as Ringer.

  • LIKE 'M[^c]%' searches for all names beginning with the letter M that do not have the letter c as the second letter, such as MacFeather.

The following query finds all employees with Engine in their job title, which would include both workers and management in “Engineering”.:

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, Title
FROM DimEmployee
WHERE Title LIKE '%Engine%';

You can use NOT LIKE with the same wildcards. To find all employees without Engine in their job title, use:

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, Title
FROM DimEmployee
WHERE Title NOT LIKE '%Engine%';

The IS NOT NULL clause can be used with wildcards and the LIKE clause. For example, the following query retrieves employees in which the job title includes Engine and IS NOT NULL:

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, Title
FROM DimEmployee
WHERE Title LIKE '%Engine%' AND Title IS NOT NULL;

Wildcards used without LIKE or PATINDEX are interpreted as constants instead of as a pattern; they represent only their own values. The following query tries to find any title that consist of the twelver characters Engineering% only. It will not find titles that start with Engineering. For more information about constants, see Constants.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, Title
FROM DimEmployee
WHERE Title = 'Engineering%';

When string comparisons are performed with LIKE or PATINDEX, all characters in the pattern string are significant, including every leading and trailing blank (space). If a comparison to return all rows with a pattern LIKE 'abc ' (abc followed by a single space) is requested, a row in which the value of that column is abc (abc without a space) is not returned. The reverse, however, is not true. Trailing blanks in the expression to which the pattern is matched are ignored. If a comparison to return all rows with a string LIKE 'abc' (abc without a space) is requested, all rows that start with abc and have zero or more trailing blanks are returned.

A string comparison using a pattern that contains char compared to varchar data (or nchar compared to nvarchar data) may not match as expected because of how the data is stored. You should understand the storage for each data type and where a LIKE comparison may fail.

Pattern Matching and Performance

An important consideration in using wildcards is their effect on performance. If a wildcard begins the expression, an index cannot be used. (Analogously, you would not know where to start in a phone book if given the partial name '%mith', not 'Smith'.) A wildcard in or at the end of an expression does not preclude use of an index. (Continuing the analogy, you would know where to search if the name was 'Samuel%'.)

Searching for Wildcard Characters

You can search for wildcard characters. Use square brackets ([ ]) to enclose the wildcard by itself. For example, to search for an underscore (_), instead of using it to specify a single character, place the underscore inside a set of brackets:

Copy image Copy Code
WHERE ColumnA LIKE 'a[_]d'

The following table shows the use of wildcards enclosed in square brackets.

Symbol | Meaning
LIKE '5[%]' | 5%
LIKE '[_]n' | _n
LIKE '[a-cdf]' | a, b, c, d, or f
LIKE '[-acdf]' | -, a, c, d, or f
LIKE '[ [ ]' | [
LIKE ']' | ]

ASCII and Unicode Pattern Matching

SQL Server PDW supports ASCII pattern matching and Unicode pattern matching. When all arguments (match_expression, pattern, and escape_character, if present) are ASCII character data types, ASCII pattern matching is performed. If any one of the arguments are of Unicode data type, all arguments are converted to Unicode and Unicode pattern matching is performed.

When you use Unicode data (nchar or nvarchar data types) in pattern matching, trailing blanks are significant; however, for non-Unicode data, trailing blanks are not significant.

The following is a series of examples that show the differences in rows returned between ASCII and Unicode LIKE pattern matching. In some examples, a trim function is used to ensure padded spaces (as with nchar types) do not affect the comparison.

Copy image Copy Code
-- ASCII pattern matching with char column
CREATE TABLE t (col1 char(30));
INSERT INTO t VALUES ('Robert King');
SELECT *
FROM t
WHERE col1 LIKE '% King'   -- returns 1 row
-- Unicode pattern matching with nchar column
CREATE TABLE t (col1 nchar(30));
INSERT INTO t VALUES ('Robert King');
SELECT *
FROM t
WHERE col1 LIKE '% King'   -- no rows returned
-- Unicode pattern matching with nchar column and RTRIM
CREATE TABLE t (col1 nchar (30));
INSERT INTO t VALUES ('Robert King');
SELECT *
FROM t
WHERE RTRIM(col1) LIKE '% King'   -- returns 1 row

Examples

A. Using LIKE with the % wildcard character

The following example finds all employees in the DimEmployee table with telephone numbers that start with 612.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, Phone
FROM DimEmployee
WHERE phone LIKE '612%'
ORDER by LastName;

B. Using NOT LIKE with the % wildcard character

The following example finds all telephone numbers in the DimEmployee table that do not start with 612. .

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, Phone
FROM DimEmployee
WHERE phone NOT LIKE '612%'
ORDER by LastName;

C. Using LIKE with the _ wildcard character

The following example finds all telephone numbers that have an area code starting with 6 and ending in 2 in the DimEmployee table. Note that the % wildcard character is also included at the end of the search pattern since the area code is the first part of the phone number and additional characters exist after in the column value.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, Phone
FROM DimEmployee
WHERE phone LIKE '6_2%'
ORDER by LastName;

D. Using the [ ] wildcard characters

The following example finds DimEmployee rows with the first name of Rob or Bob.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, Phone
FROM DimEmployee
WHERE FirstName LIKE '[RB]ob'
ORDER by LastName;

See Also

Reference

PATINDEX (SQL Server PDW)
Expressions (SQL Server PDW)
Functions (SQL Server PDW)
HAVING (SQL Server PDW)
WHERE (SQL Server PDW)


NOT (SQL Server PDW)

[image]

NOT (SQL Server PDW)
See Also Send Feedback

Negates a Boolean input.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

[ NOT ] boolean_expression

Arguments

boolean_expression

Is any valid Boolean expression. For more information, see Expressions (SQL Server PDW).

Result Types

Boolean

Result Value

NOT reverses the value of any Boolean expression.

Remarks

Using NOT negates an expression.

The following table shows the results of comparing TRUE and FALSE values using the NOT operator.

 | NOT
TRUE | FALSE
FALSE | TRUE
UNKNOWN | UNKNOWN

Examples

The following example restricts results to SalesOrderNumber to values starting with SO6 and ProductKeys greater than or equal to 400.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT ProductKey, CustomerKey, OrderDateKey, ShipDateKey
FROM FactInternetSales
WHERE SalesOrderNumber LIKE 'SO6%' AND NOT ProductKey < 400;

See Also

Reference

Expressions (SQL Server PDW)


NULL and UNKNOWN (SQL Server PDW)

[image]

NULL and UNKNOWN (SQL Server PDW)
See Also Send Feedback

NULL indicates that the value is unknown. A null value is different from an empty or zero value. No two null values are equal. Comparisons between two null values, or between a null value and any other value, return unknown because the value of each NULL is unknown.

Null values generally indicate data that is unknown, not applicable, or to be added later. For example, a customer's middle initial may not be known at the time the customer places an order.

Note the following about null values:

  • To test for null values in a query, use IS NULL or IS NOT NULL in the WHERE clause.

  • Null values can be inserted into a column by explicitly stating NULL in an INSERT or UPDATE statement or by leaving a column out of an INSERT statement.

  • Null values cannot be used as information that is required to distinguish one row in a table from another row in a table, such as primary keys, or for information used to distribute rows, such as distribution keys.

When null values are present in data, logical and comparison operators can potentially return a third result of UNKNOWN instead of just TRUE or FALSE. This need for three-valued logic is a source of many application errors. These tables outline the effect of introducing null comparisons.

The following table shows the results of applying an AND operator to two Boolean operands where one operand returns NULL.

Operand 1 | Operand 2 | Result
TRUE | NULL | FALSE
NULL | NULL | FALSE
FALSE | NULL | FALSE

The following table shows the results of applying an OR operator to two Boolean operands where one operand returns NULL.

Operand 1 | Operand 2 | Result
TRUE | NULL | TRUE
NULL | NULL | UNKNOWN
FALSE | NULL | UNKNOWN

See Also

Reference

AND (SQL Server PDW)
OR (SQL Server PDW)
NOT (SQL Server PDW)
IS [NOT] NULL (SQL Server PDW)


OR (SQL Server PDW)

[image]

OR (SQL Server PDW)
See Also Send Feedback

Combines two conditions. When more than one logical operator is used in a statement, OR operators are evaluated after AND operators. However, you can change the order of evaluation by using parentheses.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

boolean_expression OR boolean_expression

Arguments

boolean_expression

Any valid expressions that returns TRUE, FALSE, or UNKNOWN. For more information about the UNKNOWN return value, see NULL and UNKNOWN (SQL Server PDW).

Result Types

Boolean

Result Value

OR returns TRUE when either of the conditions is TRUE.

Remarks

The following table shows the result of the OR operator.

Expression1 | Expression2 | Result
TRUE | TRUE | TRUE
TRUE | FALSE | TRUE
TRUE | UNKNOWN | UNKNOWN
FALSE | UNKNOWN | UNKNOWN
FALSE | FALSE | FALSE

Examples

The following example retrieves the names of employees who either earn a BaseRate less than 20 or have a HireDate January 1, 2001 or later.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, BaseRate, HireDate
FROM DimEmployee
WHERE BaseRate < 10 OR HireDate >= '2001-01-01';

See Also

Reference

AND (SQL Server PDW)
Expressions (SQL Server PDW)
Functions (SQL Server PDW)
Operators (SQL Server PDW)
SELECT (SQL Server PDW)
WHERE (SQL Server PDW)


Bitwise Operators (SQL Server PDW)

[image]

Bitwise Operators (SQL Server PDW)
See Also Send Feedback

Bitwise operators perform bit manipulations between two expressions of any of the data types of the integer data type category.

Operator | Meaning
& (Bitwise AND) (SQL Server PDW) | Bitwise AND (two operands).
| (Bitwise OR) (SQL Server PDW) | Bitwise OR (two operands).
^ (Bitwise Exclusive OR) (SQL Server PDW) | Bitwise exclusive OR (two operands).

The operands for bitwise operators can be any one of the data types of the integer or binary string data type categories (except for the image data type), except that both operands cannot be any one of the data types of the binary string data type category. The following table shows the supported operand data types.

Left operand | Right operand
binary | int, smallint, or tinyint
bit | int, smallint, tinyint, or bit
int | int, smallint, tinyint, binary, or varbinary
smallint | int, smallint, tinyint, binary, or varbinary
tinyint | int, smallint, tinyint, binary, or varbinary
varbinary | int, smallint, or tinyint

For more information about data types, see Data Types (SQL Server PDW).

See Also

Reference

Operators (SQL Server PDW)


& (Bitwise AND) (SQL Server PDW)

[image]

& (Bitwise AND) (SQL Server PDW)
See Also Send Feedback

Performs a bitwise logical AND operation between two integer values.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression & expression

Arguments

expression

Is any valid expression of any of the data types of the integer data type category, or the bit, or the binary or varbinary data types. expression is treated as a binary number for the bitwise operation.

Note Note

In a bitwise operation, only one expression can be of either binary or varbinary data type.

Result Types

int if the input values are int.

smallint if the input values are smallint.

tinyint if the input values are tinyint or bit.

Remarks

The & bitwise operator performs a bitwise logical AND between the two expressions, taking each corresponding bit for both expressions. The bits in the result are set to 1 if and only if both bits (for the current bit being resolved) in the input expressions have a value of 1; otherwise, the bit in the result is set to 0.

If the left and right expressions have different integer data types (for example, the left expression is smallint and the right expression is int), the argument of the smaller data type is converted to the larger data type. In this case, the smallint expression is converted to an int.

Examples

The following example creates a table using the int data type to store the values and inserts two values into one row.

Copy image Copy Code
CREATE TABLE bitwise
(
a_int_value int NOT NULL,
b_int_value int NOT NULL
);
GO
INSERT bitwise VALUES (170, 75);
GO

This query performs the bitwise AND between the a_int_value and b_int_value columns.

Copy image Copy Code
SELECT a_int_value & b_int_value
FROM bitwise;
GO

Here is the result set:

Copy image Copy Code
-----------
10

(1 row(s) affected)

The binary representation of 170 (a_int_value or A) is 0000 0000 1010 1010. The binary representation of 75 (b_int_value or B) is 0000 0000 0100 1011. Performing the bitwise AND operation on these two values produces the binary result 0000 0000 0000 1010, which is decimal 10.

Copy image Copy Code
(A & B)
0000 0000 1010 1010
0000 0000 0100 1011
-------------------
0000 0000 0000 1010

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
Bitwise Operators (SQL Server PDW)
&= (Bitwise AND EQUALS) (SQL Server PDW)


| (Bitwise OR) (SQL Server PDW)

[image]

| (Bitwise OR) (SQL Server PDW)
See Also Send Feedback

Performs a bitwise logical OR operation between two specified integer values as translated to binary expressions within Transact-SQL statements.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression | expression

Arguments

expression

Is any valid expression of the integer data type category, or the bit, binary, or varbinary data types. expression is treated as a binary number for the bitwise operation.

Note Note

Only one expression can be of either binary or varbinary data type in a bitwise operation.

Result Types

Returns an int if the input values are int, a smallint if the input values are smallint, or a tinyint if the input values are tinyint.

Remarks

The bitwise | operator performs a bitwise logical OR between the two expressions, taking each corresponding bit for both expressions. The bits in the result are set to 1 if either or both bits (for the current bit being resolved) in the input expressions have a value of 1; if neither bit in the input expressions is 1, the bit in the result is set to 0.

If the left and right expressions have different integer data types (for example, the left expression is smallint and the right expression is int), the argument of the smaller data type is converted to the larger data type. In this example, the smallint expression is converted to an int.

Examples

The following example creates a table with int data types to show the original values and puts the table into one row.

Copy image Copy Code
CREATE TABLE bitwise
(
 a_int_value int NOT NULL,
b_int_value int NOT NULL
);
GO
INSERT bitwise VALUES (170, 75);
GO

The following query performs the bitwise OR on the a_int_value and b_int_value columns.

Copy image Copy Code
SELECT a_int_value | b_int_value
FROM bitwise;
GO

Here is the result set.

Copy image Copy Code
-----------
235

(1 row(s) affected)

The binary representation of 170 (a_int_value or A, below) is 0000 0000 1010 1010. The binary representation of 75 (b_int_value or B, below) is 0000 0000 0100 1011. Performing the bitwise OR operation on these two values produces the binary result 0000 0000 1110 1011, which is decimal 235.

Copy image Copy Code
(A | B)
0000 0000 1010 1010
0000 0000 0100 1011
-------------------
0000 0000 1110 1011

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
Bitwise Operators (SQL Server PDW)
|= (Bitwise OR EQUALS) (SQL Server PDW)


^ (Bitwise Exclusive OR) (SQL Server PDW)

[image]

^ (Bitwise Exclusive OR) (SQL Server PDW)
See Also Send Feedback

Performs a bitwise exclusive OR operation between two integer values.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression ^ expression

Arguments

expression

Is any valid expression of any one of the data types of the integer data type category, or the bit, or the binary or varbinary data types. expression is treated as a binary number for the bitwise operation.

Note Note

Only one expression can be of either binary or varbinary data type in a bitwise operation.

Result Types

int if the input values are int.

smallint if the input values are smallint.

tinyint if the input values are tinyint.

Remarks

The ^ bitwise operator performs a bitwise logical exclusive OR between the two expressions, taking each corresponding bit for both expressions. The bits in the result are set to 1 if either (but not both) bits (for the current bit being resolved) in the input expressions have a value of 1. If both bits are 0 or both bits are 1, the bit in the result is cleared to a value of 0.

If the left and right expressions have different integer data types (for example, the left expression is smallint and the right expression is int), the argument of the smaller data type is converted to the larger data type. In this case, the smallint expression is converted to an int.

Examples

The following example creates a table using the int data type to store the original values and inserts two values into one row.

Copy image Copy Code
CREATE TABLE bitwise
(
a_int_value int NOT NULL,
b_int_value int NOT NULL
);
GO
INSERT bitwise VALUES (170, 75);
GO

The following query performs the bitwise exclusive OR on the a_int_value and b_int_value columns.

Copy image Copy Code
SELECT a_int_value ^ b_int_value
FROM bitwise;
GO

Here is the result set:

Copy image Copy Code
-----------
225

(1 row(s) affected)

The binary representation of 170 (a_int_value or A) is 0000 0000 1010 1010. The binary representation of 75 (b_int_value or B) is 0000 0000 0100 1011. Performing the bitwise exclusive OR operation on these two values produces the binary result 0000 0000 1110 0001, which is decimal 225.

Copy image Copy Code
(A ^ B)
         0000 0000 1010 1010
         0000 0000 0100 1011
         -------------------
         0000 0000 1110 0001

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
Bitwise Operators (SQL Server PDW)
^= (Bitwise Exclusive OR EQUALS) (SQL Server PDW)


~ (Bitwise NOT) (SQL Server PDW)

[image]

~ (Bitwise NOT) (SQL Server PDW)
See Also Send Feedback

Performs a bitwise logical NOT operation on an integer value.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

~ expression

Arguments

expression

Is any valid expression of any one of the data types of the integer data type category, the bit, or the binary or varbinary data types. expression is treated as a binary number for the bitwise operation.

Note Note

Only one expression can be of either binary or varbinary data type in a bitwise operation.

Result Types

int if the input values are int.

smallint if the input values are smallint.

tinyint if the input values are tinyint.

bit if the input values are bit.

Remarks

The ~ bitwise operator performs a bitwise logical NOT for the expression, taking each bit in turn. If expression has a value of 0, the bits in the result set are set to 1; otherwise, the bit in the result is cleared to a value of 0. In other words, ones are changed to zeros and zeros are changed to ones.

Important note Important

When you perform any kind of bitwise operation, the storage length of the expression used in the bitwise operation is important. We recommend that you use this same number of bytes when storing values. For example, storing the decimal value of 5 as a tinyint, smallint, or int produces a value stored with different numbers of bytes: tinyint stores data using 1 byte; smallint stores data using 2 bytes, and int stores data using 4 bytes. Therefore, performing a bitwise operation on an int decimal value can produce different results from those using a direct binary or hexadecimal translation, especially when the ~ (bitwise NOT) operator is used. The bitwise NOT operation may occur on a variable of a shorter length. In this case, when the shorter length is converted to a longer data type variable, the bits in the upper 8 bits may not be set to the expected value. We recommend that you convert the smaller data type variable to the larger data type, and then perform the NOT operation on the result.

Examples

The following example creates a table using the int data type to store the values and inserts the two values into one row.

Copy image Copy Code
CREATE TABLE bitwise
(
a_int_value int NOT NULL,
b_int_value int NOT NULL
);
GO
INSERT bitwise VALUES (170, 75);
GO

The following query performs the bitwise NOT on the a_int_value and b_int_value columns.

Copy image Copy Code
SELECT ~ a_int_value, ~ b_int_value
FROM bitwise;

Here is the result set:

Copy image Copy Code
--- ---
-171  -76

(1 row(s) affected)

The binary representation of 170 (a_int_value or A) is 0000 0000 1010 1010. Performing the bitwise NOT operation on this value produces the binary result 1111 1111 0101 0101, which is decimal -171. The binary representation for 75 is 0000 0000 0100 1011. Performing the bitwise NOT operation produces 1111 1111 1011 0100, which is decimal -76.

Copy image Copy Code
 (~A)
         0000 0000 1010 1010
         -------------------
         1111 1111 0101 0101
(~B)
         0000 0000 0100 1011
         -------------------
         1111 1111 1011 0100

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
Bitwise Operators (SQL Server PDW)


^= (Bitwise Exclusive OR EQUALS) (SQL Server PDW)

[image]

^= (Bitwise Exclusive OR EQUALS) (SQL Server PDW)
See Also Send Feedback

Performs a bitwise exclusive OR operation between two integer values, and sets a value to the result of the operation.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression ^= expression

Arguments

expression

Is any valid expression of any one of the data types in the numeric category except the bit data type.

Result Types

Returns the data type of the argument with the higher precedence. For more information, see Data Type Precedence (Transact-SQL).

Remarks

For more information, see ^ (Bitwise Exclusive OR) (SQL Server PDW).

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
Bitwise Operators (SQL Server PDW)
^= (Bitwise Exclusive OR EQUALS) (SQL Server PDW)


|= (Bitwise OR EQUALS) (SQL Server PDW)

[image]

|= (Bitwise OR EQUALS) (SQL Server PDW)
See Also Send Feedback

Performs a bitwise logical OR operation between two specified integer values as translated to binary expressions within Transact-SQL statements, and sets a value to the result of the operation.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression |= expression

Arguments

expression

Is any valid expression of any one of the data types in the numeric category except the bit data type.

Result Types

Returns the data type of the argument with the higher precedence. For more information, see Data Type Precedence (Transact-SQL).

Remarks

For more information, see | (Bitwise OR) (SQL Server PDW).

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
Bitwise Operators (SQL Server PDW)
^= (Bitwise Exclusive OR EQUALS) (SQL Server PDW)


&= (Bitwise AND EQUALS) (SQL Server PDW)

[image]

&= (Bitwise AND EQUALS) (SQL Server PDW)
See Also Send Feedback

Performs a bitwise logical AND operation between two integer values, and sets a value to the result of the operation.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

expression &= expression

Arguments

expression

Is any valid expression of any one of the data types in the numeric category except the bit data type.

Result Types

Returns the data type of the argument with the higher precedence. For more information, see Data Type Precedence (Transact-SQL).

Remarks

For more information, see & (Bitwise AND) (SQL Server PDW).

See Also

Reference

Expressions (SQL Server PDW)
Operators (SQL Server PDW)
Bitwise Operators (SQL Server PDW)
^= (Bitwise Exclusive OR EQUALS) (SQL Server PDW)


Functions (SQL Server PDW)

[image]

Functions (SQL Server PDW)
See Also Send Feedback

SQL Server PDW provides many built-in functions that you can use to perform operations such as calculations and string manipulations. System functions can be used in most places where expressions are allowed, including a SELECT list or a WHERE clause. In situations where a system function is not supported, the statement will return an error. SQL includes the following functions:

Contents

  • Window Functions

    • Aggregate Functions

    • Analytic Functions

    • Ranking Functions

  • Scalar Functions

    • Casting Functions

    • Cryptographic Functions

    • Date and Time Functions

    • Error Handling Functions

    • Mathematical Functions

    • Metadata Functions

    • Null and Comparison Functions

    • ODBC Scalar Functions

    • String Functions

Window Functions

Window functions can be applied to row groups (windows). An OVER clause can be applied to aggregate, analytic, and ranking functions

Aggregate Functions

Aggregate functions perform a calculation on a set of values and return a scalar value. Except for COUNT, aggregate functions ignore null values. Aggregate functions are frequently used with the GROUP BY clause of the SELECT statement.

Aggregate functions can be used as expressions only in the select list and HAVING clauses of the SELECT statement.

The following table lists the aggregate functions in SQL Server PDW.

AVG | MAX
COUNT | MIN
COUNT_BIG | SUM

Analytic Functions

Analytic functions perform an aggregate calculation on a group of rows in a result set and return a scalar numeric value. The group of rows is called a window; analytic functions are also referred to as windowing functions. Analytic functions enable you to calculate rankings and percentiles, perform moving window calculations, and lag or lead analysis.

Analytic functions can be used as expressions only in the select list or HAVING clause of a SELECT statement. An analytic function is evaluated over the initial result set produced after the FROM, WHERE, GROUP BY, and HAVING clauses of the query have been evaluated. Only an ORDER BY clause applied to the full query (not the analytic function) is run after an analytic function.

The syntactic characteristic that differentiates analytic functions from all other functions is the presence of an OVER() clause. The OVER() clause serves two purposes. First, for functions that have both an analytic and an aggregate form (for example, SUM), the OVER() clause indicates that the analytic form is being invoked. Second, the contents of the OVER() clause specify how the analytic function computation itself behaves. The OVER() clause can contain an optional PARTITION BY clause and an optional ORDER BY clause. Not all analytic functions support these optional clauses. For example, the ROW_NUMBER function requires an ORDER BY clause; however, the COUNT function, in comparison, does not support an ORDER BY clause. More than one analytic function can be used in a single query. When more than one analytic function is specified, the OVER() clause for each function can differ in partitioning and also ordering.

Restrictions: In the presence of an OVER() clause as a windowing function, the following restrictions apply.

The following functions require the ORDER BY clause and are valid as a windowing function in both PDW and SQL Server.

  • LAG

  • LEAD

  • PERCENTILE_COUNT1

  • PERCENTILE_DISC1

  • MIN

  • MAX

  • STDEV

  • STDEVP

  • VAR

  • VARP

  • AVG

  • SUM

The following functions cannot be used in PDW with the ORDER BY clause as a windowing function.

  • COUNT

  • COUNT_BIG

1PERCENTILE_CONT and PERCENTILE_DISC require an ORDER BY with a single data value in the WITHIN GROUP clause. They do not support ORDER BY in the OVER() clause. The result of the percentile analytic is calculated from the domain of the ORDER BY data value (either discretely or continuously). PERCENTILE_CONT and PERCENTILE_DISC without an OVER() clause are aggregate functions that are not supported by SQL Server PDW or SQL Server.

In OVER clause for windowing aggregates, PDW supports PARTITION BY clause and RANGE clause, however PDW does not support ROWS clause. SQL Server 2012 and newer supports all that and also supports ROWS range with some limitations

Ranking Functions

Ranking functions return a ranking value for each row in a partition. Depending on the function that is used, some rows might receive the same value as other rows. Ranking functions are nondeterministic.

DENSE_RANK | NTILE | ROW_NUMBER
ROW_NUMBER |  |

Scalar Functions

Scalar functions return the same results each time they are called with a specific set of input values.

Casting Functions

Casting functions convert a value of one data type to another data type, typically for presentation or comparison against a value of another data type.

The following table lists the casting functions in SQL Server PDW.

CAST and CONVERT

Cryptographic Functions

The following table lists the cryptographic functions in SQL Server PDW.

HASHBYTES

Date and Time Functions

The following tables list the date and time functions in SQL Server PDW.

DATEADD | DAY | SYSDATETIME
DATEDIFF | EOMONTH | SYSDATETIMEOFFSET
DATEFROMPARTS | GETDATE | SYSUTCDATETIME
DATENAME | GETUTCDATE | TIMEFROMPARTS
DATEPART | ISDATE | TODATETIMEOFFSET
DATETIME2FROMPARTS | MONTH | YEAR
DATETIMEFROMPARTS | SMALLDATETIMEFROMPARTS |
DATETIMEOFFSETFROMPARTS | SWITCHOFFSET |

Error Handling Functions

The following table lists the error handling functions in SQL Server PDW. Also, see PRINT, RAISERROR, THROW, TRY...CATCH.

@@ERROR | ERROR_PROCEDURE | ERROR_STATE
ERROR_MESSAGE | ERROR_SEVERITY | XACT_STATE
ERROR_NUMBER |  |

Mathematical Functions

Mathematical functions perform a calculation based on input values that are provided as arguments and return a scalar numeric value. Scalar means that they return the same results each time they are called with a specific set of input values. Arithmetic functions, such as ABS, FLOOR, and SIGN, return a value having the same data type as the input value. Trigonometric and other functions such as EXP, LOG, LOG10, and SQRT, cast their input values to the float type and return a float value.

The following table lists the mathematical functions in SQL Server PDW.

ABS | DEGREES | RADIANS
ACOS | EXP | ROUND
ASIN | FLOOR | SIGN
ATAN | LOG | SIN
ATN2 | LOG10 | SQUARE
CEILING | PI | SQRT
COS | POWER | TAN
COT |  |

Metadata Functions

Metadata functions return a scalar value providing information about the SQL Server PDW software and objects. The following table lists the metadata functions in SQL Server PDW.

COL_NAME | OBJECT_ID | SESSION_USER
COLLATIONPROPERTY | OBJECT_NAME | SQL_VARIANT_PROPERTY
CURRENT_USER | OBJECTPROPERTY | SYSTEM_USER
DATABASEPROPERTYEX | OBJECTPROPERTYEX | TYPE_ID
DATALENGTH | SCHEMA_ID | TYPE_NAME
DB_ID (SQL Server PDW) | SCHEMA_NAME | TYPEPROPERTY
DB_NAME | SERVERPROPERTY | VERSION
INDEXPROPERTY | SESSION_ID |

Null and Comparison Functions

Null and comparison functions perform a calculation based on input values that are provided as arguments and return a scalar numeric value. The following table lists the null and comparison functions in SQL Server PDW.

CASE

ISNULL

NULLIF

ODBC Scalar Functions

ODBC scalar functions can be called from Transact-SQL. The supported ODBC scalar functions are: BIT_LENGTH, CONCAT, CUR_DATE, CUR_TIME, CURRENT_DATE, CURRENT_TIME, DAYNAME, DAYOFMONTH, DAYOFWEEK, HOUR, MINUTE, MONTHNAME, QUARTER, SECOND, and WEEK. For more information, see ODBC Scalar Functions (SQL Server PDW).

String Functions

String functions perform an operation on a string input value and return a scalar string or numeric value. All built-in string functions are deterministic. This means they return the same value any time they are called with a specific set of input values. The following table lists the string functions in SQL Server PDW.

ASCII | LTRIM | RTRIM
CHAR | NCHAR | SOUNDEX
CHARINDEX | PARSENAME | SPACE
CONCAT | PATINDEX | STUFF
DIFFERENCE | REPLACE | SUBSTRING
LEFT | REPLICATE | TERTIARY_WEIGHTS
LEN | REVERSE | UNICODE
LOWER | RIGHT | UPPER

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


@@DATEFIRST (SQL Server PDW)

[image]

@@DATEFIRST (SQL Server PDW)
See Also Send Feedback

Returns the current value of SET DATEFIRST, which specifies the first day of the week. In this release, the return value is always 7, which is Sunday, the U.S. English default.

For more information, see the @@DATEFIRST (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

@@DATEFIRST

Return Type

tinyint

Permissions

All logins, database users and database roles can access @@DATEFIRST.

Example

Copy image Copy Code
SELECT @@DATEFIRST;

See Also


@@ERROR (SQL Server PDW)

[image]

@@ERROR (SQL Server PDW)
See Also Send Feedback

Returns the error number for the last Transact-SQL statement executed. @@ERROR is a global variable.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

@@ERROR

Return Types

integer

Remarks

Returns 0 if the previous statement encountered no errors.

Returns an error number if the previous statement encountered an error.

Because @@ERROR is cleared and reset on each statement executed, check it immediately following the statement being verified, or save it to a local variable that can be checked later.

Use the TRY...CATCH construct to handle errors. The TRY...CATCH construct also supports additional system functions (ERROR_MESSAGE, ERROR_PROCEDURE, ERROR_SEVERITY, and ERROR_STATE) that return more error information than @@ERROR. TRY...CATCH also supports an ERROR_NUMBER function that is not limited to returning the error number in the statement immediately after the statement that generated an error. For more information, see TRY...CATCH (SQL Server PDW).

Examples

A. Using @@ERROR to return an error number

The following example uses @@ERROR to return the error generated by a failed data type conversion.

Copy image Copy Code
DECLARE @myint int;
SET @myint = 'ABC';
GO
SELECT 'Error number was: ', @@ERROR;
GO

See Also

Reference

TRY...CATCH (SQL Server PDW)
ERROR_MESSAGE (SQL Server PDW)
ERROR_NUMBER (SQL Server PDW)
ERROR_PROCEDURE (SQL Server PDW)
ERROR_SEVERITY (SQL Server PDW)
ERROR_STATE (SQL Server PDW)
THROW (SQL Server PDW)


@@LANGUAGE (SQL Server PDW)

[image]

@@LANGUAGE (SQL Server PDW)
See Also Send Feedback

Returns the name of the language currently being used.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

@@LANGUAGE

Return Types

nvarchar

Examples

The following example returns the language for the current session.

Copy image Copy Code
SELECT @@LANGUAGE AS 'Language Name';

Here is the result set.

Copy image Copy Code
Language Name
------------------------------
us_english

See Also


@@SPID (SQL Server PDW)

[image]

@@SPID (SQL Server PDW)
See Also Send Feedback

Returns the server process ID (SPID) for SMP SQL Server running on the SQL Server PDW Control node.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

@@SPID

Return Types

smallint

General Remarks

The @@SPID function does not return a SQL Server PDW SESSION_ID (SQL Server PDW); it returns the server process ID (SPID) for SMP SQL Server running on the Control node. This behavior is slightly different from SMP SQL Server for which SESSION_ID(), @@SPID, and the SQL Server DMVs all refer to the same type of SQL Server session.

The SPID in SQL Server is not guaranteed to be unique and can be re-used from closed sessions. For more information, see the @@SPID (Transact-SQL) documentation on Technet.

Metadata

These are examples of queries that use @@SPID in SQL Server PDW.

Copy image Copy Code
SELECT * FROM sys.dm_pdw_exec_connections WHERE sql_spid = @@SPID;
SELECT * FROM sys.dm_pdw_exec_sessions WHERE sql_spid = @@SPID;
SELECT * FROM sys.dm_pdw_exec_requests WHERE session_id = SESSION_ID();

Examples

This example returns the SQL Server PDW session ID, the SQL Server Control node session ID, login name, and user name for the current user process.

Copy image Copy Code
SELECT SESSION_ID() AS ID, @@SPID AS 'Control ID', SYSTEM_USER AS 'Login Name', USER AS 'User Name';

See Also

Reference

SESSION_ID (SQL Server PDW)


@@TRANCOUNT (SQL Server PDW)

[image]

@@TRANCOUNT (SQL Server PDW)
See Also Send Feedback

@@TRANCOUNT returns the number of BEGIN TRANSACTION statements that have occurred on the current connection in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

@@TRANCOUNT

Return Types

integer

Remarks

The BEGIN TRANSACTION statement increments @@TRANCOUNT by 1. ROLLBACK TRANSACTION decrements @@TRANCOUNT to 0. COMMIT TRANSACTION and COMMIT WORK decrement @@TRANCOUNT by 1.

General Remarks

Since SQL Server PDW does not support nested transactions, @@TRANCOUNT is always 1 or 0.

See Also


@@Version (SQL Server PDW)

[image]

@@Version (SQL Server PDW)
See Also Send Feedback

Returns version, processor architecture, build date, and operating system for the current installation of SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

@@VERSION

Return Types

nvarchar

General Remarks

The @@VERSION results are returned as one nvarchar string.

Examples

A. Return the current version of SQL Server PDW

Copy image Copy Code
SELECT @@VERSION AS 'SQL Server PDW Version';

See Also


ABS (SQL Server PDW)

[image]

ABS (SQL Server PDW)
See Also Send Feedback

Returns the absolute (positive) value of the specified numeric expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ABS ( numeric_expression )

Arguments

numeric_expression

An expression of the exact numeric or approximate numeric data type category, except for the bit data type.

Return Types

Returns the same type as numeric_expression.

Error Handling

The ABS function can produce an overflow error when the absolute value of a number is larger than the largest number that can be represented by the specified data type. For example, the int data type can hold only values that range from -2,147,483,648 to 2,147,483,647.

Examples

The following example shows the results of using the ABS function on three different numbers.

Copy image Copy Code
SELECT ABS(-1.0), ABS(0.0), ABS(1.0);

Here is the result set.

---- ---- ----

1.0 0.0 1.0

See Also

Reference

Expressions (SQL Server PDW)
CAST and CONVERT (SQL Server PDW)
Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


ACOS (SQL Server PDW)

[image]

ACOS (SQL Server PDW)
See Also Send Feedback

A mathematical function that returns the angle, in radians, whose cosine is the specified float expression; also called arccosine.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ACOS ( float_expression )

Arguments

float_expression

Is an expression of the type float or of a type that can be implicitly converted to float, with a value from -1 through 1. Values outside this range return NULL and report a domain error.

Return Types

float

Examples

The following example returns the ACOS of the specified number.

Copy image Copy Code
DECLARE @cos float;
SET @cos = -1.0;
SELECT 'The ACOS of the number is: ' + CONVERT(varchar, ACOS(@cos));

Here is the result set.

Copy image Copy Code
---------------------------------
The ACOS of the number is: 3.14159

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


ASCII (SQL Server PDW)

[image]

ASCII (SQL Server PDW)
See Also Send Feedback

Returns the ASCII code value of the leftmost character of a character expression.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ASCII ( character_expression )

Arguments

character_expression

Is an expression of the type char or varchar.

Return Types

int

Examples

The following example assumes an ASCII character set and returns the ASCII value for 6 characters.

Copy image Copy Code
SELECT ASCII('A') AS A, ASCII('B') AS B,
ASCII('a') AS a, ASCII('b') AS b,
ASCII(1) AS [1], ASCII(2) AS [2];

Here is the result set.

Copy image Copy Code
A           B           a           b           1           2
----------- ----------- ----------- ----------- ----------- -----------
65          66          97          98          49          50

See Also

Reference

Functions (SQL Server PDW)
CHAR (SQL Server PDW)


ASIN (SQL Server PDW)

[image]

ASIN (SQL Server PDW)
See Also Send Feedback

Returns the angle, in radians, whose sine is the specified float expression in SQL Server PDW. This is also referred to as the arcsine.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ASIN ( float_expression )

Arguments

float_expression

An expression of the type float or of a type that can be implicitly converted to float, with a value from -1 through 1. Values outside this range return NULL and report a domain error.

Return Types

float

Examples

The following example returns the arcsine of 1.00.

Copy image Copy Code
SELECT ASIN(1.00) AS asinCalc;

The following example returns an error, because it requests the arcsine for a value outside the allowed range.

Copy image Copy Code
SELECT ASIN(1.1472738) AS asinCalc;

See Also

Reference

Expressions (SQL Server PDW)
SIN (SQL Server PDW)
Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


ATAN (SQL Server PDW)

[image]

ATAN (SQL Server PDW)
See Also Send Feedback

Returns the angle in radians whose tangent is a specified float expression in SQL Server PDW. This is also referred to as the arctangent.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ATAN ( float_expression )

Arguments

float_expression

Is an expression of type float or of a type that can be implicitly converted to float.

Return Types

float

Examples

The following example takes a float expression and returns the arctangent of the specified angle.

Copy image Copy Code
SELECT ATAN(45.87) AS atanCalc1,
    ATAN(-181.01) AS atanCalc2,
    ATAN(0) AS atanCalc3,
    ATAN(0.1472738) AS atanCalc4,
    ATAN(197.1099392) AS atanCalc5;

Here is the result set.

atanCalc1 atanCalc2 atanCalc3 atanCalc4 atanCalc5

--------- --------- --------- --------- ---------

1.55 -1.57 0.00 0.15 1.57

See Also

Reference

Expressions (SQL Server PDW)
TAN (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


ATN2 (SQL Server PDW)

[image]

ATN2 (SQL Server PDW)
See Also Send Feedback

Returns the angle, in radians, between the positive x-axis and the ray from the origin to the point (y, x), where x and y are the values of the two specified float expressions.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ATN2 ( float_expression , float_expression )

Arguments

float_expression

Is an expression of the float data type.

Return Types

float

Examples

The following example calculates the ATN2 for the specified x and y components.

Copy image Copy Code
DECLARE @x float = 35.175643, @y float = 129.44;
SELECT 'The ATN2 of the angle is: ' + CONVERT(varchar,ATN2(@x,@y ));
GO

Here is the result set.

Copy image Copy Code
The ATN2 of the angle is: 0.265345
(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


AVG (SQL Server PDW)

[image]

AVG (SQL Server PDW)
See Also Send Feedback

Returns the average of the values in a group in SQL Server PDW. Use this function in the select list or HAVING clause of a SELECT statement.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Aggregate Function Syntax
AVG ( [ ALL | DISTINCT ] expression )
-- Analytic Function Syntax
AVG (expression) OVER ( [ <partition_by_clause> ] [ <order_by_clause> ] )

Arguments

ALL

Applies the aggregate function to all values. ALL is the default.

DISTINCT

Averages only each unique instance of a value, regardless of how many times the value occurs.

expression

An expression of the exact numeric or approximate numeric data type category, except for the bit data type. Aggregate functions, analytic functions, and subqueries are not permitted.

OVER

Determines the partitioning and ordering of the rowset before the analytic function is applied.

< partition_by_clause >

Divides the result set produced by the FROM clause into partitions to which the AVG function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

< order_by_clause >

Determines the order in which the AVG calculation is applied to the rows in a partition. An integer cannot represent a column in order_by_clause. When the order_by_clause is absent, all rows in the same partition (as specified by the partition_by_clause) are assigned the same value for the AVG() OVER computation. When the order_by_clause is present, the AVG() OVER computation gives you a cumulative average in the partition. Within the same partitions, rows with identical values for all elements of the order_by_clause return identical results for the AVG() OVER computation.

Return Types

The return type is determined by the type of the evaluated result of expression.

Expression Result | Return Type
tinyint | int
smallint | int
int | int
bigint | bigint
decimal category (p, s) | decimal(38, s) divided by decimal(10, 0)
float | float

Interoperability

A GROUP BY clause must be included in the query if selecting an aggregate value (such as AVG) and at least one column or value that is not an aggregate function.

Limitations and Restrictions

The AVG() OVER analytic function does not support the use of the ALL or DISTINCT modifiers.

AVG() computes the average of a set of values by dividing the sum of those values by the count of non-null values. If the sum exceeds the maximum value for the data type of the return value an error will be returned.

Examples

A. Using the SUM and AVG functions for calculations

The following example calculates the average vacation hours and the sum of sick leave hours that the vice presidents of a company have used. Each of these aggregate functions produces a single summary value for all the retrieved rows.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT AVG(VacationHours)AS AverageVacationHours,
SUM(SickLeaveHours) AS TotalSickLeaveHours
FROM dbo.DimEmployee
WHERE Title LIKE 'Vice President%';

Here is the result set.

AverageVacationHours TotalSickLeaveHours

-------------------- --------------------------

25 97

B. Using the SUM and AVG functions together with a GROUP BY clause

The following example uses the GROUP BY clause to return a single aggregate value for each group, instead of for the whole table. The example produces summary values for each sales territory. The summary lists the average sales by the sales representatives in each territory and the sum of sales for each territory.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT SalesTerritoryKey AS Territory,
    CONVERT(varchar(12), AVG(SalesAmount),1) AS AverageSales,
CONVERT(varchar(15), SUM(SalesAmount),1) AS TotalSales
FROM dbo.FactResellerSales
WHERE SalesTerritoryKey BETWEEN 1 and 4
GROUP BY SalesTerritoryKey;

Here is the result set.

Territory AverageSales TotalSales

----------- ------------ ---------------

1 1,579.66 12,435,076.00

2 1,193.47 6,932,842.01

3 1,360.29 7,906,008.18

4 1,380.26 18,466,458.79

C. Using AVG with DISTINCT

The following example returns the average list price in the Product table. The function averages only each unique instance of ListPrice.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT AVG(DISTINCT ListPrice) FROM dbo.DimProduct;

Here is the result set.

----------------

512.2459

D. Using AVG without DISTINCT

The following example returns the average list price of all products in the Product table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT AVG(ListPrice) FROM dbo.DimProduct;

Here is the result set.

----------------

747.6617

E. Using AVG with OVER

The following examples show using the OVER clause with the AVG function to return the cumulative average price of products within each product line.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT AVG(ListPrice)OVER(PARTITION BY ProductLine)AS AvgPrice
,ProductLine
FROM dbo.DimProduct
WHERE ListPrice IS NOT NULL
      AND ProductLine IS NOT NULL
GROUP BY ProductLine,ListPrice
ORDER BY AvgPrice;

Here is the result set.

AvgPrice ProductLine

------------ ---------------

40.5016 S

414.8843 T

555.1098 R

617.2562 M

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
ORDER BY (SQL Server PDW)


CASE (SQL Server PDW)

[image]

CASE (SQL Server PDW)
See Also Send Feedback

Evaluates a list of conditions and returns one of multiple possible result expressions in SQL Server PDW.

CASE can be used in any statement or clause that allows a valid expression. For example, you can use CASE in statements such as SELECT, UPDATE, DELETE and SET, and in clauses such as select_list, IN, WHERE, ORDER BY, and HAVING.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CASE
     WHEN when_expression THEN result_expression [ ...n ]
     [ ELSE else_result_expression ]
END

Arguments

WHEN when_expression

Any valid Boolean expression.

THEN result_expression

The expression returned when when_expression evaluates to TRUE. result expression is any valid expression.

ELSE else_result_expression

The expression returned if no comparison operation evaluates to TRUE. If this argument is omitted and no comparison operation evaluates to TRUE, CASE returns NULL. else_result_expression is any valid expression. The data types of else_result_expression and any result_expression must be the same or must be an implicit conversion.

Return Types

Returns the highest precedence type from the set of types in result_expressions and the optional else_result_expression. For more information, see Data Type Precedence in Data Types (SQL Server PDW).

Limitations and Restrictions

SQL Server PDW does not allow for nesting in CASE expressions.

Examples

A. Using a SELECT statement with a CASE expression

Within a SELECT statement, the CASE expression allows for values to be replaced in the result set based on comparison values. The following example uses the CASE expression to change the display of product line categories to make them more understandable. When a value does not exist, the text “Not for sale’ is displayed.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT   ProductAlternateKey, Category =
      CASE ProductLine
         WHEN 'R' THEN 'Road'
         WHEN 'M' THEN 'Mountain'
         WHEN 'T' THEN 'Touring'
         WHEN 'S' THEN 'Other sale items'
         ELSE 'Not for sale'
      END,
   EnglishProductName
FROM dbo.DimProduct
ORDER BY ProductKey;

B. Using CASE in an UPDATE statement

The following example uses the CASE expression in an UPDATE statement to determine the value that is set for the column VacationHours for employees with SalariedFlag set to 0. When subtracting 10 hours from VacationHours results in a negative value, VacationHours is increased by 40 hours; otherwise, VacationHours is increased by 20 hours.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

UPDATE dbo.DimEmployee
SET VacationHours =
    ( CASE
         WHEN ((VacationHours - 10.00) < 0) THEN VacationHours + 40
         ELSE (VacationHours + 20.00)
       END
    )
WHERE SalariedFlag = 0;

See Also

Reference

Expressions (SQL Server PDW)
SELECT (SQL Server PDW)
UPDATE (SQL Server PDW)
WHERE (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


CAST and CONVERT (SQL Server PDW)

[image]

CAST and CONVERT (SQL Server PDW)
See Also Send Feedback

Converts an expression of one data type to another data type in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

--Syntax for CAST
CAST ( expression AS data_type [ (length ) ])

--Syntax for CONVERT
CONVERT ( data_type [ ( length ) ] , expression [ , style ] )

Arguments

expression

A valid expression.

data_type

The target data type.

length

An optional integer that specifies the length of the target data type. The default value is 30.

style

An integer expression that specifies how the CONVERT function is to translate expression. If style is NULL, NULL is returned. The range is determined by data_type.

Return Types

Returns expression translated to data_type.

Data Types

Date and Time

When expression is a date or time data type, style can be one of the values shown in the following table. Other values are processed as 0. SQL Server PDW supports the date format in Arabic style by using the Kuwaiti algorithm.

Without century (yy) (1) | With century (yyyy) | Standard | Input/Output (3)
- | 0 or 100 (1, 2) | Default for datetime and smalldatetime | mon dd yyyy hh:miAM (or PM)
1 | 101 | U.S. | mm/dd/yyyy
2 | 102 | ANSI | yy.mm.dd
3 | 103 | British/French | dd/mm/yyyy
4 | 104 | German | dd.mm.yy
5 | 105 | Italian | dd-mm-yy
6 | 106 (1) | - | dd mon yy
7 | 107 (1) | - | mon dd, yy
8 | 108 | - | hh:mi:ss
- | 9 or 109 (1, 2) | Default + milliseconds | mon dd yyyy hh:mi:ss:mmmAM (or PM)
10 | 110 | USA | mm-dd-yy
11 | 111 | JAPAN | yy/mm/dd
12 | 112 | ISO | yymmdd yyyymmdd
- | 13 or 113 (1, 2) | Europe default + milliseconds | dd mon yyyy hh:mi:ss:mmm(24h)
14 | 114 | - | hh:mi:ss:mmm(24h)
- | 20 or 120 (2) | ODBC canonical | yyyy-mm-dd hh:mi:ss(24h)
- | 21 or 121 (2) | ODBC canonical (with milliseconds) default for time, date, datetime2, and datetimeoffset | yyyy-mm-dd hh:mi:ss.mmm(24h)
- | 126 (4) | ISO8601 | yyyy-mm-ddThh:mi:ss.mmm (no spaces)
- | 127(6, 7) | ISO8601 with time zone Z. | yyyy-mm-ddThh:mi:ss.mmmZ (no spaces)
- | 130 (1, 2) | Hijri (5) | dd mon yyyy hh:mi:ss:mmmAM
- | 131 (2) | Hijri (5) | dd/mm/yy hh:mi:ss:mmmAM

1 These style values return nondeterministic results. Includes all (yy) (without century) styles and a subset of (yyyy) (with century) styles.

2 The default values (style 0 or 100, 9 or 109, 13 or 113, 20 or 120, and 21 or 121) always return the century (yyyy).

3 Input when you convert to datetime; output when you convert to character data.

4 For conversion from datetime to character data, the output format is as described in the previous table.

5 Hijri is a calendar system with several variations. SQL Server PDW uses the Kuwaiti algorithm.

Important note Important

By default, SQL Server PDW interprets two-digit years based on a cutoff year of 2049. That is, the two-digit year 49 is interpreted as 2049 and the two-digit year 50 is interpreted as 1950. Many client applications, such as those based on Automation objects, use a cutoff year of 2030. SQL Server PDW provides the two digit year cutoff configuration option that changes the cutoff year used by SQL Server PDW and allows for the consistent treatment of dates. We recommend specifying four-digit years.

6 Only supported when casting from character data to datetime. When character data that represents only date or only time components is cast to the datetime data types, the unspecified time component is set to 00:00:00.000, and the unspecified date component is set to 1900-01-01.

7The optional time zone indicator, Z, is used to make it easier to map XML datetime values that have time zone information to SQL Server PDW datetime values that have no time zone. Z is the indicator for time zone UTC-0. Other time zones are indicated with HH:MM offset in the + or - direction. For example: 2006-12-12T23:45:12-08:00.

You can truncate unwanted date parts when you convert from datetime values by using an appropriate char or varchar data type length.

float

When expression is float, style can be one of the values shown in the following table. Other values are processed as 0.

Value | Output
0 (default) | A maximum of 6 digits. Use in scientific notation, when appropriate.
1 | Always 8 digits. Always use in scientific notation.
2 | Always 16 digits. Always use in scientific notation.

General Remarks

When you convert character expressions (char, nchar, nvarchar, or varchar) to an expression of a different data type, data can be truncated, only partially displayed, or an error is returned because the result is too short to display. Conversions to char, nchar, nvarchar or varchar are truncated, except for the conversions shown in the following table.

From data type | To data type | Result
int, smallint, or tinyint | char | *
 | varchar | *
 | nchar | E
 | nvarchar | E
decimal or float | char | E
 | varchar | E
 | nchar | E
 | nvarchar | E

* = Result length too short to display. E = Error returned because result length is too short to display.

SQL Server PDW guarantees that only "roundtrip conversions"--conversions that convert a data type from its original data type and back again--will yield the same values from version to version. The following example shows such a roundtrip conversion:

Copy image Copy Code
SELECT TOP(1) CAST(CAST('193.57' AS varchar(20)) AS decimal(10,5)) FROM dbo.DimCustomer;

or

Copy image Copy Code
SELECT TOP(1) CONVERT(decimal(10,5), CONVERT(varchar(20), '193.57')) FROM dbo.DimCustomer;

The following example shows a resulting expression that is too small to display.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName,
       CAST(SickLeaveHours AS char(1)) AS SickLeave
FROM dbo.DimEmployee
WHERE NOT EmployeeKey > 5
AND EndDate IS NULL;

Here is the result set.

FirstName LastName Sick Leave

--------------- --------------- ----------

Guy Gilbert *

Kevin Brown *

Roberto Tamburello *

Rob Walters *

When you convert data types that differ in decimal places, sometimes the result value is truncated and at other times it is rounded. The following table shows the behavior.

From | To | Behavior
float | int | Truncate
float | datetime | Round
datetime | int | Round

For example, the result of the following conversion is 10:

SELECT TOP(1) CAST(10.6496 AS int) FROM dbo.DimCustomer;

When you convert data types in which the target data type has fewer decimal places than the source data type, the value is rounded. For example, the result of the following conversion is 10.3497:

SELECT TOP(1) CAST(10.3496847 AS decimal (10, 4)) FROM dbo.DimCustomer;

SQL Server PDW returns an error message when nonnumeric char, nchar, varchar, or nvarchar data is converted to int, float, or decimal.

Implicit Conversions

Implicit conversions are those conversions that occur without specifying the CAST or CONVERT functions. Explicit conversions are those conversions that require the CAST or CONVERT function to be specified. See "Data Type Conversion" in Data Types (SQL Server PDW) for information on all explicit and implicit data type conversions that are allowed for SQL Server PDW data types.

Examples

A. Using CAST and CONVERT

This example retrieves the name of the product for those products that have a 3 in the first digit of their list price and converts their ListPrice to int.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EnglishProductName AS ProductName, ListPrice
FROM dbo.DimProduct
WHERE CAST(ListPrice AS int) LIKE '3%';

This example shows the same query, using CONVERT instead of CAST.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EnglishProductName AS ProductName, ListPrice
FROM dbo.DimProduct
WHERE CONVERT(int, ListPrice) LIKE '3%';

B. Using CAST with arithmetic operators

The following example calculates a single column computation by dividing the product unit price (UnitPrice) by the discount percentage (UnitPriceDiscountPct). This result is converted to an int data type after being rounded to the nearest whole number.

Copy image Copy Code
USE
    AdventureWorksPDW2012

SELECT ProductKey, UnitPrice,UnitPriceDiscountPct,
       CAST(ROUND (UnitPrice*UnitPriceDiscountPct,0) AS int) AS DiscountPrice
FROM dbo.FactResellerSales
WHERE SalesOrderNumber = 'SO47355'
      AND UnitPriceDiscountPct > .02

Here is the result set.

ProductKey UnitPrice UnitPriceDiscountPct DiscountPrice

---------- --------- -------------------- -------------

323 430.6445 0.05 22

213 18.5043 0.05 1

456 37.4950 0.10 4

456 37.4950 0.10 4

216 18.5043 0.05 1

C. Using CAST to concatenate

The following example concatenates noncharacter expressions by using CAST.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT 'The list price is ' + CAST(ListPrice AS varchar(12)) AS ListPrice
FROM dbo.DimProduct
WHERE ListPrice BETWEEN 350.00 AND 400.00;

Here is the result set.

ListPrice

------------------------

The list price is 357.06

The list price is 364.09

The list price is 364.09

The list price is 364.09

The list price is 364.09

D. Using CAST to produce more readable text

The following example uses CAST in the SELECT list to convert the Name column to a char(10) column.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT CAST(EnglishProductName AS char(10)) AS Name, ListPrice
FROM dbo.DimProduct
WHERE EnglishProductName LIKE 'Long-Sleeve Logo Jersey, M';

Here is the result set.

Name UnitPrice

---------- ---------

Long-Sleev 31.2437

Long-Sleev 32.4935

Long-Sleev 49.99

E. Using CAST with the LIKE clause

The following example converts the money column ListPrice to an int type and then to a char(20) type so that it can be used with the LIKE clause.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EnglishProductName AS Name, ListPrice
FROM dbo.DimProduct
WHERE CAST(CAST(ListPrice AS int) AS char(20)) LIKE '2%';

G. Using CAST and CONVERT with datetime data

The following example displays the current date and time, uses CAST to change the current date and time to a character data type, and then uses CONVERT display the date and time in the ISO 8601 format.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1)
   SYSDATETIME() AS UnconvertedDateTime,
   CAST(SYSDATETIME() AS nvarchar(30)) AS UsingCast,
   CONVERT(nvarchar(30), SYSDATETIME(), 126) AS UsingConvertTo_ISO8601
FROM dbo.DimCustomer;

Here is the result set.

UnconvertedDateTime UsingCast UsingConvertTo_ISO8601

----------------------- ------------------------------ ------------------------------

07/20/2010 1:44:31 PM 2010-07-20 13:44:31.5879025 2010-07-20T13:44:31.5879025

The following example is approximately the opposite of the previous example. The example displays a date and time as character data, uses CAST to change the character data to the datetime data type, and then uses CONVERT to change the character data to the datetime data type.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1)
   '2010-07-25T13:50:38.544' AS UnconvertedText,
CAST('2010-07-25T13:50:38.544' AS datetime) AS UsingCast,
   CONVERT(datetime, '2010-07-25T13:50:38.544', 126) AS UsingConvertFrom_ISO8601
FROM dbo.DimCustomer;

Here is the result set.

UnconvertedText UsingCast UsingConvertFrom_ISO8601

----------------------- ----------------------- ------------------------

2010-07-25T13:50:38.544 07/25/2010 1:50:38 PM 07/25/2010 1:50:38 PM

See Also

Reference

Expressions (SQL Server PDW)
SELECT (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


CEILING (SQL Server PDW)

[image]

CEILING (SQL Server PDW)
See Also Send Feedback

Returns the smallest integer greater than, or equal to, the specified numeric expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CEILING ( numeric_expression )

Arguments

numeric_expression

An expression of the exact numeric or approximate numeric data type category, except for the bit data type.

Return Types

Returns the same type as numeric_expression.

Examples

The following example shows use of positive numeric, negative, and zero values with the CEILING function.

Copy image Copy Code
SELECT CEILING(123.45), CEILING(-123.45), CEILING(0.0);

Here is the result set.

------- --------- --------

124.00 -123.00 0.00

See Also

Reference

Expressions (SQL Server PDW)
FLOOR (SQL Server PDW)


CHAR (SQL Server PDW)

[image]

CHAR (SQL Server PDW)
See Also Send Feedback

Converts an int ASCII code to a character.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CHAR ( integer_expression )

Arguments

integer_expression

Is an integer from 0 through 255. NULL is returned if the integer expression is not in this range.

Return Types

char(1)

Remarks

CHAR can be used to insert control characters into character strings. The following table shows some frequently used control characters.

Control character | Value
Tab | char(9)
Line feed | char(10)
Carriage return | char(13)

Examples

A. Using ASCII and CHAR to print ASCII values from a string

The following example assumes an ASCII character set and returns the character value for 6 ASCII character numbers.

Copy image Copy Code
SELECT CHAR(65) AS [65], CHAR(66) AS [66],
CHAR(97) AS [97], CHAR(98) AS [98],
CHAR(49) AS [49], CHAR(50) AS [50];

Here is the result set.

Copy image Copy Code
65   66   97   98   49   50
---- ---- ---- ---- ---- ----
A    B    a    b    1    2

B. Using CHAR to insert a control character

The following example uses CHAR(13) to return information about the databases on separate lines when the results are returned in text.

Copy image Copy Code
SELECT name, 'was created on ', create_date, CHAR(13), name, 'is currently ', state_desc
FROM sys.databases;
GO

Here is the result set.

Copy image Copy Code
name     create_date    name    state_desc
------------------------------------------------------------
master                   was created on  2003-04-08 09:13:36.390
master                   is currently  ONLINE
tempdb                   was created on  2014-01-10 17:24:24.023
tempdb                   is currently  ONLINE
AdventureWorksPDW2012    was created on  2014-05-07 09:05:07.083 AdventureWorksPDW2012    is currently  ONLINE

See Also

Reference

Functions (SQL Server PDW)
ASCII (SQL Server PDW)


CHARINDEX (SQL Server PDW)

[image]

CHARINDEX (SQL Server PDW)
See Also Send Feedback

Returns the first occurrence of one expression within another expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CHARINDEX ( string_pattern , string_expression , start_location)

Arguments

string_pattern

The substring to be found. string_pattern must be of a character data type. string_pattern cannot be an empty string ('').

string_expression

The string expression to be searched. string_expression must be of a character data type.

start_location

An int or bigint expression at which the search starts. The first position in the string is '1'. If start_location is not specified, is a negative number, or is 0, the search starts at the beginning of string_expression.

Return Types

smallint

General Remarks

The first position in a string is position one ('1'). The location of string_pattern is based on the location of the first character in the string.

If either string_pattern or string_expression is of a Unicode data type (nvarchar or nchar) and the other is not, the other is converted to a Unicode data type.

When string_pattern is not found, CHARINDEX returns '0'.

Examples

A. Searching from the start of a string expression

The following example returns the first location of the is string in This is a string, starting from position 1 (the first character) in the string.

Copy image Copy Code
SELECT CHARINDEX('is', 'This is a string');

Here is the result set.

---------

3

B. Searching from a position other than the first position

The following example returns the first location of the is string in This is a string, starting with the fourth position.

Copy image Copy Code
SELECT CHARINDEX('is', 'This is a string', 4);

Here is the result set.

---------

6

C. Results when the string is not found

The following example shows the return value when the string_pattern is not found in the searched string.

Copy image Copy Code
SELECT TOP(1) CHARINDEX('at', 'This is a string') FROM dbo.DimCustomer;

Here is the result set.

---------

0

See Also

Reference

Expressions (SQL Server PDW)
PATINDEX (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


COALESCE (SQL Server PDW)

[image]

COALESCE (SQL Server PDW)
See Also Send Feedback

Returns the first non-null expression among its arguments in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

COALESCE ( expression [ ,...n ] )

Arguments

expression

An expression of any type.

Return Types

Returns the data type of expression with the highest data type precedence. If all expressions are non-nullable, the result is typed as non-nullable. If all arguments are null, COALESCE returns null. At least one of the null values must be a typed null.

General Remarks

COALESCE(expression1,...n) is equivalent to the following CASE expression:

Copy image Copy Code
CASE
   WHEN (expression1 IS NOT NULL) THEN expression1
   ...
   WHEN (expressionN IS NOT NULL) THEN expressionN
   ELSE NULL
END

ISNULL and COALESCE, though closely related, can behave differently. An expression involving ISNULL with non-null parameters is considered to be NOT NULL, while expressions involving COALESCE with non-null parameters is considered to be NULL.

Examples

Simple Example

The following example demonstrates how COALESCE selects the data from the first column that has a non-null value. Assume for this example that the Products table contains this data:

Name Color ProductNumber

------------ ---------- -------------

Socks, Mens NULL PN1278

Socks, Mens Blue PN1965

NULL White PN9876

We then run the following COALESCE query:

Copy image Copy Code
SELECT Name, Color, ProductNumber, COALESCE(Color, ProductNumber) AS FirstNotNull
FROM Products ;

Here is the result set.

Name Color ProductNumber FirstNotNull

------------ ---------- ------------- ------------

Socks, Mens NULL PN1278 PN1278

Socks, Mens Blue PN1965 Blue

NULL White PN9876 White

Notice that in the first row, the FirstNotNull value is “PN1278”, not “Socks, Mens”. This is because the Name column was not specified as a parameter for COALESCE in the example.

Complex Example

The following example uses COALESCE to compare the values in three columns and return only the non-null value found in the columns.

Copy image Copy Code
CREATE TABLE dbo.wages
(
    emp_id        tinyint   NULL,
    hourly_wage   decimal   NULL,
    salary        decimal   NULL,
    commission    decimal   NULL,
    num_sales     tinyint   NULL
);
INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (1, 10.00, NULL, NULL, NULL);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (2, 20.00, NULL, NULL, NULL);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (3, 30.00, NULL, NULL, NULL);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (4, 40.00, NULL, NULL, NULL);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (5, NULL, 10000.00, NULL, NULL);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (6, NULL, 20000.00, NULL, NULL);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (7, NULL, 30000.00, NULL, NULL);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (8, NULL, 40000.00, NULL, NULL);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (9, NULL, NULL, 15000, 3);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (10,NULL, NULL, 25000, 2);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (11, NULL, NULL, 20000, 6);

INSERT INTO dbo.wages (emp_id, hourly_wage, salary, commission, num_sales)
VALUES (12, NULL, NULL, 14000, 4);

SELECT CAST(COALESCE(hourly_wage * 40 * 52,
   salary,
   commission * num_sales) AS decimal(10,2)) AS TotalSalary
FROM dbo.wages
ORDER BY TotalSalary;

Here is the result set.

Total Salary

------------

10000.00

20000.00

20800.00

30000.00

40000.00

41600.00

45000.00

50000.00

56000.00

62400.00

83200.00

120000.00

See Also

Reference

Expressions (SQL Server PDW)
CASE (SQL Server PDW)
CAST and CONVERT (SQL Server PDW)


COL_NAME (SQL Server PDW)

[image]

COL_NAME (SQL Server PDW)
Send Feedback

Returns the name of a column from a specified corresponding table identification number and column identification number. For more information, see the COL_NAME (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

COL_NAME ( table_id , column_id  )

Arguments

table_id

The identification number of the table that contains the column. table_id is of type int.

column_id

The identification number of the column. column_id parameter is of type int.

Return Types

sysname

Exceptions

Returns NULL on error or if a caller does not have permission to view the object.

A user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as COL_NAME may return NULL if the user does not have any permission on the object.

Remarks

The table_id and column_id parameters together produce a column name string.

For more information about obtaining table and column identification numbers, see OBJECT_ID.

Examples

The following example returns the name of the first column in a sample Employee table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
GO
SELECT COL_NAME(OBJECT_ID('dbo.FactResellerSales'), 1) AS FirstColumnName,
COL_NAME(OBJECT_ID('dbo.FactResellerSales'), 2) AS SecondColumnName;

Here is the result set.

Copy image Copy Code
ColumnName
------------
BusinessEntityID


COLLATIONPROPERTY (SQL Server PDW)

[image]

COLLATIONPROPERTY (SQL Server PDW)
See Also Send Feedback

Returns the property of a specified collation. For more information, see the COLLATIONPROPERTY (Transact-SQL)documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

COLLATIONPROPERTY( collation_name , property )

Arguments

collation_name

Is the name of the collation. collation_name is nvarchar(128), and has no default.

property

Is the property of the collation. property is varchar(128), and can be any one of the following values:

Property name

Description

CodePage

Non-Unicode code page of the collation.

LCID

Windows LCID of the collation.

ComparisonStyle

Windows comparison style of the collation. Returns 0 for all binary collations .

Version

The version of the collation, derived from the version field of the collation ID. Returns 2, 1, or 0.

Collations with "100" in the name) return 2.

Collations with "90" in the name) return 1.

All other collations return 0.

PDW supports Windows only collations and this function works for all PDW supported collations. Constant literals, variables, column references, and arbitrary expressions can be used as input parameters.

Return Types

sql_variant

Examples

Copy image Copy Code
SELECT COLLATIONPROPERTY('Traditional_Spanish_CS_AS_KS_WS', 'CodePage')

Here is the result set.

Copy image Copy Code
1252

See Also

Concepts

Collations (SQL Server PDW)


CONCAT (SQL Server PDW)

[image]

CONCAT (SQL Server PDW)
See Also Send Feedback

Returns a string that is the result of concatenating two or more string values.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CONCAT ( string_value1, string_value2 [, string_valueN ] )

Arguments

string_value

A string value to concatenate to the other values.

Return Types

String, the length and type of which depend on the input.

Remarks

CONCAT takes a variable number of string arguments and concatenates them into a single string. It requires a minimum of two input values; otherwise, an error is raised. All arguments are implicitly converted to string types and then concatenated. Null values are implicitly converted to an empty string. If all the arguments are null, an empty string of type varchar(1) is returned. The implicit conversion to strings follows the existing rules for data type conversions. For more information about data type conversions, see CAST and CONVERT (SQL Server PDW).

The return type depends on the type of the arguments. The following table illustrates the mapping.

Input type | Output type and length
If any argument is a SQL-CLR system type, a SQL-CLR UDT, or nvarchar(max) | nvarchar(max)
Otherwise, if any argument is varbinary(max) or varchar(max) | varchar(max) unless one of the parameters is an nvarchar of any length. If so, then the result is nvarchar(max).
Otherwise, if any argument is nvarchar(<= 4000) | nvarchar(<= 4000)
Otherwise, in all other cases | varchar(<= 8000)unless one of the parameters is an nvarchar of any length. If so, then the result is nvarchar(max).

When the arguments are <= 4000 for nvarchar, or <= 8000 for varchar, implicit conversions can affect the length of the result. Other data types have different lengths when they are implicitly converted to strings. For example, an int (14) has a string length of 12, while a float has a length of 32. Thus the result of concatenating two integers has a length of no less than 24.

If none of the input arguments is of a supported large object (LOB) type, then the return type is truncated to 8000 in length, regardless of the return type. This truncation preserves space and supports efficiency in plan generation.

Examples

A. Using CONCAT

Copy image Copy Code
SELECT CONCAT ( 'Happy ', 'Birthday ', 11, '/', '25' ) AS Result;

Here is the result set.

Copy image Copy Code
Result
-------------------------
Happy Birthday 11/25

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


COS (SQL Server PDW)

[image]

COS (SQL Server PDW)
See Also Send Feedback

Returns the trigonometric cosine of the specified angle, in radians, represented by the specified expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

COS ( float_expression )

Arguments

float_expression

Is an expression of type float.

Return Types

float

Examples

The following example returns the COS of the specific angle.

Copy image Copy Code
SELECT COS(14.76) AS cosCalc1, COS(-0.1472738) AS cosCalc2;

Here is the result set.

cosCalc1 cosCalc2

-------- --------

-0.58 0.99

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


COT (SQL Server PDW)

[image]

COT (SQL Server PDW)
See Also Send Feedback

A mathematical function that returns the trigonometric cotangent of the specified angle, in radians, in the specified float expression.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

COT ( float_expression )

Arguments

float_expression

Is an expression of type float or of a type that can be implicitly converted to float.

Return Types

float

Examples

The following example returns the COT for the specific angle.

Copy image Copy Code
DECLARE @angle float;
SET @angle = 124.1332;
SELECT 'The COT of the angle is: ' + CONVERT(varchar,COT(@angle));
GO

Here is the result set.

Copy image Copy Code
The COT of the angle is: -0.040312

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


COUNT (SQL Server PDW)

[image]

COUNT (SQL Server PDW)
See Also Send Feedback

Returns the number of items in a group in SQL Server PDW. Use this function in the select list or HAVING clause of a SELECT statement to aggregate the number of rows in a table or partition.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Aggregation Function Syntax
COUNT ( { [ [ ALL | DISTINCT ] expression ] | * } )
-- Analytic Function Syntax
COUNT ( { expression | * } ) OVER ( [ <partition_by_clause> ] )

Arguments

ALL

Applies the aggregate function to all values. COUNT (ALL expression) evaluates expression for each row in a group and returns the number of non-null values. ALL is the default.

DISTINCT

Specifies that COUNT returns the number of unique non-null values. COUNT (DISTINCT expression) evaluates expression for each row in a group or partition and returns the number of unique, non-null values.

expression

Is an expression of any type. Aggregate functions and subqueries are not permitted.

*

Specifies that all rows (including null and duplicate values) should be counted to return the total number of rows in a table or partition. COUNT(*) takes no parameters and cannot be used with ALL or DISTINCT.

OVER

Determines the partitioning and ordering of the rowset before the analytic function is applied.

< partition_by_clause >

Divides the result set produced by the FROM clause into partitions to which the COUNT function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

Return Types

int

Limitation and Restrictions

The COUNT() OVER analytic function does not support the use of the ALL or DISTINCT modifiers for the expression to compute the COUNT against.

Examples

A. Using COUNT and DISTINCT

The following example lists the number of different titles that an employee who works at a specific company can hold.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT COUNT(DISTINCT Title)
FROM dbo.DimEmployee;

Here is the result set.

-----------

67

B. Using COUNT(*)

The following example returns the total number of rows in the dbo.DimEmployee table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT COUNT(*)
FROM dbo.DimEmployee;

Here is the result set.

-------------

296

C. Using COUNT(*) with other aggregates

The following example combines COUNT(*) with other aggregate functions in the SELECT list. The query returns the number of sales representatives with a annual sales quota greater than $500,000 and the average sales quota.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT COUNT(EmployeeKey) AS TotalCount, AVG(SalesAmountQuota) AS [Average Sales Quota]
FROM dbo.FactSalesQuota
WHERE SalesAmountQuota > 500000 AND CalendarYear = 2001;

Here is the result set.

TotalCount Average Sales Quota

---------- -------------------

10 683800.0000

D. Using COUNT with HAVING

The following example uses COUNT with the HAVING clause to return the departments in a company that have more than 15 employees.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DepartmentName,
       COUNT(EmployeeKey)AS EmployeesInDept
FROM dbo.DimEmployee
GROUP BY DepartmentName
HAVING COUNT(EmployeeKey) > 15;

Here is the result set.

DepartmentName EmployeesInDept

-------------- ---------------

Sales 18

Production 179

E. Using COUNT with OVER

The following example uses COUNT with the OVER clause to return the number of products that are contained in each of the specified sales orders.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT COUNT(ProductKey) OVER(PARTITION BY SalesOrderNumber) AS ProductCount
    ,SalesOrderNumber
FROM dbo.FactInternetSales
WHERE SalesOrderNumber IN (N'SO53115',N'SO55981');

Here is the result set.

ProductCount SalesOrderID

------------ -----------------

3 SO53115

1 SO55981

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
OVER Clause (SQL Server PDW)
COUNT_BIG (SQL Server PDW)


COUNT_BIG (SQL Server PDW)

[image]

COUNT_BIG (SQL Server PDW)
See Also Send Feedback

Returns the number of items in a group in SQL Server PDW. Use this function in the select list or HAVING clause of a SELECT statement to aggregate the number of rows in a table or partition. COUNT_BIG differs from the COUNT function only in their return values. COUNT_BIG always returns a bigint data type value. COUNT always returns an int data type value.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Aggregation Function Syntax
COUNT_BIG ( { [ [ ALL | DISTINCT ] expression ] | * } )
-- Analytic Function Syntax
COUNT_BIG ( { expression | * } ) OVER ( [ <partition_by_clause> ] )

Arguments

ALL

Applies the aggregate function to all values. COUNT_BIG (ALL expression) evaluates expression for each row in a group and returns the number of non-null values. ALL is the default.

DISTINCT

Specifies that COUNT_BIG return the number of unique non-null values. COUNT_BIG (DISTINCT expression) evaluates expression for each row in a group or partition and returns the number of unique, non-null values.

expression

Is an expression of any type. Aggregate functions and subqueries are not permitted.

*

Specifies that all rows (including null and duplicate values) should be counted to return the total number of rows in a table or partition. COUNT_BIG(*) takes no parameters and cannot be used with ALL or DISTINCT.

OVER

Determines the partitioning and ordering of the rowset before the analytic function is applied.

< partition_by_clause >

Divides the result set produced by the FROM clause into partitions to which the COUNT_BIG function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

Return Types

bigint

Limitation and Restrictions

The COUNT_BIG() OVER analytic function does not support the use of the ALL or DISTINCT modifiers for the expression to compute the COUNT_BIG function against.

Examples

For examples, see COUNT (SQL Server PDW).

See Also

Reference

COUNT (SQL Server PDW)
OVER Clause (SQL Server PDW)
Expressions (SQL Server PDW)
Functions (SQL Server PDW)


CURRENT_TIMESTAMP

[image]

CURRENT_TIMESTAMP
See Also Send Feedback

Returns the current database system timestamp as a datetime value without the database time zone offset. This value is derived from the Windows operating system running on the Control node.

For more information, see CURRENT_TIMESTAMP ( Transact-SQL) on MSDN.

Topic link icon Transact-SQL Syntax Conventions

Syntax

CURRENT_TIMESTAMP

Arguments

Takes no arguments.

Return Type

datetime

Permissions

All database users have access to CURRENT_TIMESTAMP.

Examples

Copy image Copy Code
SELECT CURRENT_TIMESTAMP;

See Also


CURRENT_USER (SQL Server PDW)

[image]

CURRENT_USER (SQL Server PDW)
Send Feedback

Returns the name of the current user. This function is equivalent to USER_NAME().

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CURRENT_USER

Return Types

sysname

Remarks

CURRENT_USER returns the name of the current security context.

To return the login of the current user, see SYSTEM_USER.

Examples

The following example returns the name of the current user.

Copy image Copy Code
SELECT CURRENT_USER;


DATABASEPROPERTYEX (SQL Server PDW)

[image]

DATABASEPROPERTYEX (SQL Server PDW)
Send Feedback

Returns the current setting of the specified database option or property for the specified database. Not all SQL Server properties are supported by SQL Server PDW. For more information, see the DATABASEPROPERTYEX (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATABASEPROPERTYEX ( database , property )

Arguments

database

The name of the database for which to return the named property information. database is nvarchar(128).

property

An expression that represents the name of the database property to return. property is varchar(128), and can be one of the following values. The return type is sql_variant. The following table shows the base data type for each property value. property is not case sensitive.

Note Note

If the database is not started, properties that SQL Server PDW retrieves by accessing the database directly instead of retrieving the value from metadata will return NULL.

Property

Description

Value returned

Collation

Default collation name for the database.

Collation name

NULL = Database is not started.

Base data type: nvarchar(128)

IsQuotedIdentifiersEnabled

Double quotation marks can be used on identifiers. SQL Server PDW returns the value of the control node.

1 = TRUE

0 = FALSE

NULL = Input not valid

Base data type: int

IsTornPageDetectionEnabled

The SQL Server Database Engine detects incomplete I/O operations caused by power failures or other system outages. SQL Server PDW returns TRUE.

1 = TRUE

0 = FALSE

NULL = Input not valid

Base data type: int

SQLSortOrder

SQL Server sort order ID supported in earlier versions of SQL Server. SQL Server PDW returns 0.

0 = Database is using Windows collation

>0 = SQL Server sort order ID

NULL = Input not valid or database is not started

Base data type: tinyint

Status

Database status.

ONLINE = Database is available for query.

OFFLINE = Database was explicitly taken offline.

RESTORING = Database is being restored.

RECOVERING = Database is recovering and not yet ready for queries.

SUSPECT = Database did not recover.

EMERGENCY = Database is in an emergency, read-only state. Access is restricted to sysadmin members

Base data type: nvarchar(128)

Updateability

Indicates whether data can be modified. SQL Server PDW returns READ_WRITE.

READ_ONLY = Data can be read but not modified.

READ_WRITE = Data can be read and modified.

Base data type: nvarchar(128)

UserAccess

Indicates which users can access the database. SQL Server PDW returns MULTI_USER>

SINGLE_USER = Only one db_owner, dbcreator, or sysadmin user at a time

RESTRICTED_USER = Only members of db_owner, dbcreator, and sysadmin roles

MULTI_USER = All users

Base data type: nvarchar(128)

Version

Internal version number of the SQL Server code with which the database was created. Identified for informational purposes only. Not supported. Future compatibility is not guaranteed. SQL Server PDW returns the value of the control node.

Version number = Database is open.

NULL = Database is not started.

Base data type: int

Return Types

sql_variant

Exceptions

Returns NULL on error or if a caller does not have permission to view the object.

In SQL Server, a user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as OBJECT_ID may return NULL if the user does not have any permission on the object.

Remarks

DATABASEPROPERTYEX returns only one property setting at a time. To display multiple property settings, use the sys.databases catalog view.

Examples

Copy image Copy Code
SELECT DATABASEPROPERTYEX('AdventureWorksPDW2012', 'Collation')


DATALENGTH (SQL Server PDW)

[image]

DATALENGTH (SQL Server PDW)
See Also Send Feedback

Returns the number of bytes used to represent any expression.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATALENGTH ( expression )

Arguments

expression

Is an expression of any data type.

Return Types

bigint if expression is of the varchar(max), nvarchar(max) or varbinary(max) data types; otherwise int.

Remarks

DATALENGTH is especially useful with varchar, varbinary, text, image, nvarchar, and ntext data types because these data types can store variable-length data.

The DATALENGTH of NULL is NULL.

Examples

The following example finds the length of the Name column in the Product table.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
SELECT length = DATALENGTH(EnglishProductName), EnglishProductName
FROM dbo.DimProduct
ORDER BY EnglishProductName;
GO

See Also

Reference

Functions (SQL Server PDW)
LEN (SQL Server PDW)


DATEADD (SQL Server PDW)

[image]

DATEADD (SQL Server PDW)
See Also Send Feedback

Returns a specified date with the specified number interval (signed int) added to a specified date part of that date in SQL Server PDW. Use this function in the SELECT <list>, WHERE, HAVING, GROUP BY and ORDER BY clauses to add a specified value to a part of a date column or expression.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATEADD (datepart , number, date )

Arguments

datepart

The part of date to which an int number is added. The following table lists all valid datepart arguments. User-defined variable equivalents are not valid.

datepart

Abbreviations

year

yy, yyyy

quarter

qq, q

month

mm, m

dayofyear

dy, y

day

dd, d

week

wk, ww

weekday

dw, w

hour

hh

minute

mi, n

second

ss, s

millisecond

ms

microsecond

mcs

nanosecond

ns

number

An expression that can be resolved to an int that is added to a datepart of date. number cannot exceed the range of int. If you specify a value with a decimal fraction, the fraction is truncated and not rounded.

date

An expression that can be resolved to a time, date, smalldatetime, datetime, datetime2, or datetimeoffset value. date can be an expression, column expression, or string literal. If the expression is a string literal, it must resolve to a datetime. An error will be raised if the string literal seconds scale is more than three positions (.nnn) or contains the time zone offset part.

To avoid ambiguity, use four-digit years when specifying date.

Return Types

The return data type is the data type of the date argument, except for string literals, which must resolve to and return a datetime.

datepart Argument

dayofyear, day, and weekday return the same value.

Each datepart and its abbreviations return the same value.

If datepart is month and the date month has more days than the return month and the date day does not exist in the return month, the last day of the return month is returned. For example, September has 30 days; therefore, the two following statements return 9/30/2006 12:00:00 AM.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1) DATEADD(month, 1, '2006-08-30')
FROM DimCustomer;
SELECT TOP(1)DATEADD(month, 1, '2006-08-31')
FROM DimCustomer;

The date argument cannot be incremented to a value outside the range of its data type. In the following statements, the number value that is added to the date value exceeds the range of the date data type. The following error message is returned: "Adding a value to a 'datetime' column caused an overflow."

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1) DATEADD(year,2147483647, '2006-07-31') FROM DimCustomer;
SELECT TOP(1) DATEADD(year,-2147483647, '2006-07-31')FROM DimCustomer;

Remarks

Fractional Seconds Precision

Addition for a datepart of microsecond or nanosecond for date data types date and datetime or string literals is not allowed.

Milliseconds have a scale of 3 (.123). microseconds have a scale of 6 (.123456). nanoseconds have a scale of 9 (.123456789). The time and datetime2 data types have a maximum scale of 7 (.1234567). If datepart is nanosecond, number must be 100 before the fractional seconds of date increase. A number between 1 and 49 is rounded down to 0 and a number from 50 to 99 is rounded up to 100.

The following statements add a datepart of millisecond, microsecond, or nanosecond.

Copy image Copy Code
CREATE TABLE Customer (
  Id INT,
  CustomerDate datetime2);

INSERT INTO Customer VALUES (1, '2007-01-01 13:10:10.1111111');

SELECT '1 millisecond' ,DATEADD(millisecond,1,CustomerDate) FROM Customer
UNION ALL
SELECT '2 milliseconds', DATEADD(millisecond,2, CustomerDate) FROM Customer
UNION ALL
SELECT '1 microsecond', DATEADD(microsecond,1, CustomerDate) FROM Customer
UNION ALL
SELECT '2 microseconds', DATEADD(microsecond,2, CustomerDate) FROM Customer
UNION ALL
SELECT '49 nanoseconds', DATEADD(nanosecond,49, CustomerDate) FROM Customer
UNION ALL
SELECT '50 nanoseconds', DATEADD(nanosecond,50, CustomerDate) FROM Customer
UNION ALL
SELECT '150 nanoseconds', DATEADD(nanosecond,150, CustomerDate) FROM Customer;

Here is the result set.

----------------- ---------------------------

1 millisecond 01/01/2007 13:10:10.1121111

2 milliseconds 01/01/2007 1:10:10.1131111

1 microsecond 01/01/2007 1:10:10.1111121

2 microseconds 01/01/2007 1:10:10.1111131

49 nanoseconds 01/01/2007 1:10:10.1111111

50 nanoseconds 01/01/2007 1:10:10.1111112

150 nanoseconds 01/01/2007 1:10:10.1111113

Time Zone Offset

Addition is not allowed for time zone offset.

Examples

A. Incrementing datepart by an interval of 1

Each of the following statements increments datepart by an interval of 1.

Copy image Copy Code
CREATE TABLE Customer (
  ID INTEGER,
  CustomerDate datetime2);

INSERT INTO Customer VALUES (1, '2007-01-01 13:10:10.1111111');

SELECT 'year', DATEADD(year,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'quarter',DATEADD(quarter,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'month',DATEADD(month,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'dayofyear',DATEADD(dayofyear,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'day',DATEADD(day,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'week',DATEADD(week,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'weekday',DATEADD(weekday,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'hour',DATEADD(hour,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'minute',DATEADD(minute,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'second',DATEADD(second,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'millisecond',DATEADD(millisecond,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'microsecond',DATEADD(microsecond,1,CustomerDate) FROM Customer
UNION ALL
SELECT 'nanosecond',DATEADD(nanosecond,1,CustomerDate) FROM Customer;

-------------- --------------------------

Year 01/01/2008 1:10:10.1111111

Quarter 04/01/2007 1:10:10.1111111

Month 02/01/2007 1:10:10.1111111

Dayofyear 01/02/2007 1:10:10.1111111

Day 01/02/2007 1:10:10.1111111

Week 01/08/2007 1:10:10.1111111

Weekday 01/02/2007 1:10:10.1111111

Hour 01/01/2007 2:10:10.1111111

Minute 01/01/2007 1:11:10.1111111

Second 01/01/2007 1:10:11.1111111

Millisecond 01/01/2007 1:10:10.1121111

Microsecond 01/01/2007 1:10:10.1111121

Nanosecond 01/01/2007 1:10:10.1111111

B. Using expressions as arguments for the number and date parameters

The following examples use different types of expressions as arguments for the number and date parameters.

Specifying column as date

The following example adds 60 days to each HireDate to calculate the date that the employee is eligible for company benefits.

Copy image Copy Code
USE
    AdventureWorksPDW2012

SELECT EmployeeKey
    ,HireDate
    ,DATEADD(day,60,HireDate) AS BenefitsDate
FROM DimEmployee;

Specifying scalar system function as date

The following example specifies SYSDATETIME for date.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP (1) DATEADD(month, 1, SYSDATETIME()) FROM DimCustomer;

Specifying scalar subqueries and scalar functions as number and date

The following example uses scalar subqueries and scalar functions (MAX(HireDate)), as arguments for number and date. (SELECT (TOP(1) EmployeeKey FROM dbo.DimEmployee) is an artificial argument for the number parameter to show how to select a number argument from a value list.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1) DATEADD (month, (SELECT TOP(1) EmployeeKey FROM dbo.DimEmployee),
    (SELECT MAX(HireDate) FROM dbo.DimEmployee))
FROM dbo.DimCustomer;

Specifying constants as number and date

The following example uses numeric and character constants as arguments for number and date.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP (1) DATEADD(minute, 1, '2007-05-07 09:53:01.123') FROM DimCustomer;

Specifying numeric expressions and scalar system functions as number and date

The following example uses a numeric expressions (-(10/2)), and a scalar system function (SYSDATETIME) as arguments for number and date.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP (1) DATEADD(month,10/2, SYSDATETIME()) FROM DimCustomer;

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
DATEPART (SQL Server PDW)
DATEDIFF (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


DATEDIFF (SQL Server PDW)

[image]

DATEDIFF (SQL Server PDW)
See Also Send Feedback

Returns the count (signed integer) of the specified datepart boundaries crossed between the specified startdate and enddate in SQL Server PDW. Use this function in the select list, WHERE, HAVING, GROUP BY and ORDER BY clauses to subtract datetime values and return their difference.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATEDIFF ( datepart , startdate , enddate )

Arguments

datepart

The part of startdate and enddate that specifies the type of boundary crossed. The following table lists all valid datepart arguments.

datepart

Abbreviations

year

yy, yyyy

quarter

qq, q

month

mm, m

dayofyear

dy, y

day

dd, d

week

wk, ww

hour

hh

minute

mi, n

second

ss, s

millisecond

ms

microsecond

mcs

nanosecond

ns

startdate

An expressions that can be resolved to a time, date, smalldatetime, datetime, datetime2, or datetimeoffset value. date can be an expression, column expression, or string literal. startdate is subtracted from enddate.

To avoid ambiguity, use four-digit years.

enddate

See startdate.

Return Type

int

Return Value

Each datepart and its abbreviations return the same value.

If the return value is out of range for int (-2,147,483,648 to +2,147,483,647), an error is returned. For millisecond, the maximum difference between startdate and enddate is 24 days, 20 hours, 31 minutes and 23.647 seconds. For second, the maximum difference is 68 years.

If startdate and enddate are both assigned only a time value and the datepart is not a time datepart, 0 is returned.

A time zone offset component of startdate or endate is not used in calculating the return value.

Because smalldatetime is accurate only to the minute, when a smalldatetime value is used for startdate or enddate, seconds and milliseconds are always set to 0 in the return value.

If only a time value is assigned to a variable of a date data type, the value of the missing date part is set to the default value: 1900-01-01. If only a date value is assigned to a variable of a time or date data type, the value of the missing time part is set to the default value: 00:00:00. If either startdate or enddate have only a time part and the other only a date part, the missing time and date parts are set to the default values.

If startdate and enddate are of different date data types and one has more time parts or fractional seconds precision than the other, the missing parts of the other are set to 0.

datepart Boundaries

The following statements have the same startdate and the same endate. Those dates are adjacent and differ in time by .0000001 second. The difference between the startdate and endate in each statement crosses one calendar or time boundary of its datepart. Each statement returns 1. If different years are used for this example and if both startdate and endate are in the same calendar week, the return value for week would be 0.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1) DATEDIFF(year, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

SELECT TOP(1) DATEDIFF(quarter, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

SELECT TOP(1) DATEDIFF(month, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

SELECT TOP(1) DATEDIFF(dayofyear, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

SELECT TOP(1) DATEDIFF(day, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

SELECT TOP(1) DATEDIFF(week, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

SELECT TOP(1) DATEDIFF(hour, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

SELECT TOP(1) DATEDIFF(minute, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

SELECT TOP(1) DATEDIFF(second, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

SELECT TOP(1) DATEDIFF(millisecond, '2005-12-31 23:59:59.9999999'

, '2006-01-01 00:00:00.0000000') FROM DimCustomer;

Examples

The following examples use different types of expressions as arguments for the startdate and enddate parameters.

A. Specifying columns for startdate and enddate

The following example calculates the number of day boundaries that are crossed between dates in two columns in a table.

Copy image Copy Code
CREATE TABLE dbo.Duration (
    startDate datetime2
    ,endDate datetime2
    );
INSERT INTO dbo.Duration(startDate,endDate)
    VALUES('2007-05-06 12:10:09','2007-05-07 12:10:09');
SELECT TOP(1) DATEDIFF(day,startDate,endDate) AS Duration
FROM dbo.Duration;
-- Returns: 1

B. Specifying scalar subqueries and scalar functions for startdate and enddate

The following example uses scalar subqueries and scalar functions as arguments for startdate and enddate.

Copy image Copy Code
USE
    AdventureWorksPDW2012

SELECT TOP(1) DATEDIFF(day,(SELECT MIN(HireDate) FROM dbo.DimEmployee),
    (SELECT MAX(HireDate) FROM dbo.DimEmployee))
FROM dbo.DimEmployee;

C. Specifying constants for startdate and enddate

The following example uses character constants as arguments for startdate and enddate.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1) DATEDIFF(day, '2007-05-07 09:53:01.0376635'
    , '2007-05-08 09:53:01.0376635') FROM DimCustomer;

D. Specifying ranking functions for startdate

The following example uses a ranking function as an argument for startdate.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName
,DATEDIFF(day,ROW_NUMBER() OVER (ORDER BY
        DepartmentName),SYSDATETIME()) AS RowNumber
FROM dbo.DimEmployee;

E. Specifying an aggregate window function for startdate

The following example uses an aggregate window function as an argument for startdate.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, DepartmentName
    ,DATEDIFF(year,MAX(HireDate)
             OVER (PARTITION BY DepartmentName),SYSDATETIME()) AS SomeValue
FROM dbo.DimEmployee

See Also

Reference

DATEADD (SQL Server PDW)
DATEPART (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


DATEFROMPARTS (SQL Server PDW)

[image]

DATEFROMPARTS (SQL Server PDW)
See Also Send Feedback

Returns a date value for the specified year, month, and day.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATEFROMPARTS ( year, month, day )

Arguments

year

Integer expression specifying a year.

month

Integer expression specifying a month, from 1 to 12.

day

Integer expression specifying a day.

Return Types

date

Remarks

DATEFROMPARTS returns a date value with the date portion set to the specified year, month and day, and the time portion set to the default. If the arguments are not valid, then an error is raised. If required arguments are null, then null is returned.

Examples

The following example demonstrates the DATEFROMPARTS function.

Copy image Copy Code
SELECT DATEFROMPARTS ( 2010, 12, 31 ) AS Result;

Here is the result set.

Copy image Copy Code
Result
----------------------------------
2010-12-31

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


DATENAME (SQL Server PDW)

[image]

DATENAME (SQL Server PDW)
See Also Send Feedback

Returns a character string that represents the specified datepart of the specified date

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATENAME ( datepart , date )

Arguments

datepart

Is the part of the date to return. The following table lists all valid datepart arguments. User-defined variable equivalents are not valid.

datepart

Abbreviations

year

yy, yyyy

quarter

qq, q

month

mm, m

dayofyear

dy, y

day

dd, d

week

wk, ww

weekday

dw, w

hour

hh

minute

mi, n

second

ss, s

millisecond

ms

microsecond

mcs

nanosecond

ns

TZoffset

tz

ISO_WEEK

ISOWK, ISOWW

date

Is an expression that can be resolved to a time, date, smalldatetime, datetime, datetime2, or datetimeoffset value. date can be an expression, column expression, user-defined variable, or string literal.

To avoid ambiguity, use four-digit years. For information about two-digit years, see Configure the two digit year cutoff Server Configuration Option.

Return Type

nvarchar

Return Value

  • Each datepart and its abbreviations return the same value.

The return value depends on the language environment which cannot be changed. The return value is dependent on SET DATEFORMAT (SQL Server PDW) if date is a string literal of some formats. SET DATEFORMAT does not affect the return value when the date is a column expression of a date or time data type.

When the date parameter has a date data type argument, the return value depends on the setting specified by using SET DATEFIRST (SQL Server PDW).

TZoffset datepart Argument

If datepart argument is TZoffset (tz) and the date argument has no time zone offset, 0 is returned.

smalldatetime date Argument

When date is smalldatetime, seconds are returned as 00.

Default Returned for a datepart That Is Not in the date Argument

If the data type of the date argument does not have the specified datepart, the default for that datepart will be returned only when a literal is specified for date.

For example, the default year-month-day for any date data type is 1900-01-01. The following statement has date part arguments for datepart, a time argument for date, and returns 1900, January, 1, 1, Monday.

Copy image Copy Code
SELECT DATENAME(year, '12:10:30.123')
    ,DATENAME(month, '12:10:30.123')
    ,DATENAME(day, '12:10:30.123')
    ,DATENAME(dayofyear, '12:10:30.123')
    ,DATENAME(weekday, '12:10:30.123');

If date is specified as a variable or table column and the data type for that variable or column does not have the specified datepart, error 9810 is returned. The following code example fails because the date part year is not a valid for the time data type that is declared for the variable @t.

Copy image Copy Code
DECLARE @t time = '12:10:30.123';
SELECT DATENAME(year, @t);

Remarks

DATENAME can be used in the select list, WHERE, HAVING, GROUP BY, and ORDER BY clauses.

In SQL Server 2014, DATENAME implicitly casts string literals as a datetime2 type. This means that DATENAME does not support the format YDM when the date is passed as a string. You must explicitly cast the string to a datetime or smalldatetime type to use the YDM format.

Examples

The following example returns the date parts for the specified date.

Copy image Copy Code
SELECT DATENAME(datepart,'2007-10-30 12:15:32.1234567 +05:10');

Here is the result set.

datepart | Return value
year, yyyy, yy | 2007
quarter, qq, q | 4
month, mm, m | October
dayofyear, dy, y | 303
day, dd, d | 30
week, wk, ww | 44
weekday, dw | Tuesday
hour, hh | 12
minute, n | 15
second, ss, s | 32
millisecond, ms | 123
microsecond, mcs | 123456
nanosecond, ns | 123456700
TZoffset, tz | 310
ISO_WEEK, ISOWK, ISOWW | 44

See Also

Reference

CAST and CONVERT (SQL Server PDW)
Functions (SQL Server PDW)


DATEPART (SQL Server PDW)

[image]

DATEPART (SQL Server PDW)
See Also Send Feedback

Returns part of a date type value, such as the hour, year, or day in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATEPART ( date_part , date_expression )

Arguments

date_part

The part of the date or time to be retrieved. For a list of possible values, see the datepart parameter for DATEADD.

date_expression

A datetime expression of any supported format or type. For more information about supported date or time data type, see Data Types (SQL Server PDW).

Return Types

int

Remarks

When the DATEPART function is used in a WHERE clause, it forces table scanning instead of range partition query processing. This can cause suboptimal query performance.

Examples

The following example returns the day part of the date 12/20/1974.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1) DATEPART (day,'12/20/1974') FROM dbo.DimCustomer;

Here is the result set.

--------

20

The following example returns the year part of the date 12/20/1974.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1) DATEPART (year,'12/20/1974') FROM dbo.DimCustomer;

Here is the result set.

--------

1974

See Also

Reference

Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


DATETIME2FROMPARTS (SQL Server PDW)

[image]

DATETIME2FROMPARTS (SQL Server PDW)
See Also Send Feedback

Returns a datetime2 value for the specified date and time and with the specified precision.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATETIME2FROMPARTS ( year, month, day, hour, minute, seconds, fractions, precision )

Arguments

year

Integer expression specifying a year.

month

Integer expression specifying a month.

day

Integer expression specifying a day.

hour

Integer expression specifying hours.

minute

Integer expression specifying minutes.

seconds

Integer expression specifying seconds.

fractions

Integer expression specifying fractions.

precision

Integer literal specifying the precision of the datetime2 value to be returned.

Return Types

datetime2 ( precision )

Remarks

DATETIME2FROMPARTS returns a fully initialized datetime2 value. If the arguments are not valid, an error is raised. If required arguments are null, then null is returned. However, if the precision argument is null, then an error is raised.

The fractions argument depends on the precision argument. For example, if precision is 7, then each fraction represents 100 nanoseconds; if precision is 3, then each fraction represents a millisecond. If the value of precision is zero, then the value of fractions must also be zero; otherwise, an error is raised.

Examples

A. Simple example without fractions of a second

Copy image Copy Code
SELECT DATETIME2FROMPARTS ( 2010, 12, 31, 23, 59, 59, 0, 0 ) AS Result;

Here is the result set.

Copy image Copy Code
Result
---------------------------
2010-12-31 23:59:59.0000000

(1 row(s) affected)

B. Example with fractions of a second

The following example demonstrates the use of the fractions and precision parameters:

  1. When fractions has a value of 5 and precision has a value of 1, then the value of fractions represents 5/10 of a second.

  2. When fractions has a value of 50 and precision has a value of 2, then the value of fractions represents 50/100 of a second.

  3. When fractions has a value of 500 and precision has a value of 3, then the value of fractions represents 500/1000 of a second.

Transact-SQL Copy image Copy Code
SELECT DATETIME2FROMPARTS ( 2011, 8, 15, 14, 23, 44, 5, 1 );
SELECT DATETIME2FROMPARTS ( 2011, 8, 15, 14, 23, 44, 50, 2 );
SELECT DATETIME2FROMPARTS ( 2011, 8, 15, 14, 23, 44, 500, 3 );
GO

Here is the result set.

Copy image Copy Code
----------------------
2011-08-15 14:23:44.5

(1 row(s) affected)

----------------------
2011-08-15 14:23:44.50

(1 row(s) affected)

----------------------
2011-08-15 14:23:44.500

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


DATETIMEFROMPARTS (SQL Server PDW)

[image]

DATETIMEFROMPARTS (SQL Server PDW)
See Also Send Feedback

Returns a datetime value for the specified date and time.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATETIMEFROMPARTS ( year, month, day, hour, minute, seconds, milliseconds )

Arguments

year

Integer expression specifying a year.

month

Integer expression specifying a month.

day

Integer expression specifying a day.

hour

Integer expression specifying hours.

minute

Integer expression specifying minutes.

seconds

Integer expression specifying seconds.

milliseconds

Integer expression specifying milliseconds.

Return Types

datetime

Remarks

DATETIMEFROMPARTS returns a fully initialized datetime value. If the arguments are not valid, then an error is raised. If required arguments are null, then a null is returned.

Examples

Copy image Copy Code
SELECT DATETIMEFROMPARTS ( 2010, 12, 31, 23, 59, 59, 0 ) AS Result;

Here is the result set.

Copy image Copy Code
Result
---------------------------
2010-12-31 23:59:59.000

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


DATETIMEOFFSETFROMPARTS (SQL Server PDW)

[image]

DATETIMEOFFSETFROMPARTS (SQL Server PDW)
See Also Send Feedback

Returns a datetimeoffset value for the specified date and time and with the specified offsets and precision.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DATETIMEOFFSETFROMPARTS ( year, month, day, hour, minute, seconds, fractions, hour_offset, minute_offset, precision )

Arguments

year

Integer expression specifying a year.

month

Integer expression specifying a month.

day

Integer expression specifying a day.

hour

Integer expression specifying hours.

minute

Integer expression specifying minutes.

seconds

Integer expression specifying seconds.

fractions

Integer expression specifying fractions.

hour_offset

Integer expression specifying the hour portion of the time zone offset.

minute_offset

Integer expression specifying the minute portion of the time zone offset.

precision

Integer literal specifying the precision of the datetimeoffset value to be returned.

Return Types

datetimeoffset ( precision )

Remarks

DATETIMEOFFSETFROMPARTS returns a fully initialized datetimeoffset data type. The offset arguments are used to represent the time zone offset. If the offset arguments are omitted, then the time zone offset is assumed to be 00:00, that is, there is no time zone offset. If the offset arguments are specified, then both arguments must be present and both must be positive or negative. If minute_offset is specified without hour_offset, an error is raised. If other arguments are not valid, then an error is raised. If required arguments are null, then a null is returned. However, if the precision argument is null, then an error is raised.

The fractions argument depends on the precision argument. For example, if precision is 7, then each fraction represents 100 nanoseconds; if precision is 3, then each fraction represents a millisecond. If the value of precision is zero, then the value of fractions must also be zero; otherwise, an error is raised.

Examples

A. Simple example without fractions of a second

Copy image Copy Code
SELECT DATETIMEOFFSETFROMPARTS ( 2010, 12, 31, 14, 23, 23, 0, 12, 0, 7 ) AS Result;

Here is the result set.

Copy image Copy Code
Result
-------------------------------------------
2010-12-07 00:00:00.0000000 +00:00

(1 row(s) affected)

B. Example with fractions of a second

The following example demonstrates the use of the fractions and precision parameters:

  1. When fractions has a value of 5 and precision has a value of 1, then the value of fractions represents 5/10 of a second.

  2. When fractions has a value of 50 and precision has a value of 2, then the value of fractions represents 50/100 of a second.

  3. When fractions has a value of 500 and precision has a value of 3, then the value of fractions represents 500/1000 of a second.

Transact-SQL Copy image Copy Code
SELECT DATETIMEOFFSETFROMPARTS ( 2011, 8, 15, 14, 30, 00, 5, 12, 30, 1 );
SELECT DATETIMEOFFSETFROMPARTS ( 2011, 8, 15, 14, 30, 00, 50, 12, 30, 2 );
SELECT DATETIMEOFFSETFROMPARTS ( 2011, 8, 15, 14, 30, 00, 500, 12, 30, 3 );
GO

Here is the result set.

Copy image Copy Code
----------------------------------
2011-08-15 14:30:00.5 +12:30

(1 row(s) affected)

----------------------------------
2011-08-15 14:30:00.50 +12:30

(1 row(s) affected)

----------------------------------
2011-08-15 14:30:00.500 +12:30

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


DAY (SQL Server PDW)

[image]

DAY (SQL Server PDW)
See Also Send Feedback

Returns an integer representing the day (day of the month) of the specified date in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DAY ( date )

Arguments

date

An expression that can be resolved to a time, date, smalldatetime, datetime, datetime2, or datetimeoffset value. The date argument can be an expression, column expression, or string literal.

Return Type

int

Return Value

DAY returns the same value as DATEPART (day, date).

If date contains only a time part, the return value is 1, the base day.

Examples

The following example returns 30. This is the number of the day.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP 1 DAY('2010-07-30T01:01:01.1234')
FROM dbo.DimCustomer;

The following example returns 1900, 1, 1. The argument for date is the number 0. SQL Server interprets 0 as January 1, 1900.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP 1 YEAR(0), MONTH(0), DAY(0) FROM dbo.DimCustomer;

See Also

Reference

MONTH (SQL Server PDW)
YEAR (SQL Server PDW)


DB_ID (SQL Server PDW)

[image]

DB_ID (SQL Server PDW)
See Also Send Feedback

Returns the database identification (ID) number for a SQL Server PDW database.

For more information, see the DB_ID (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DB_ID ( [ 'database_name' ] );

Arguments

'database_name'

Is the database name for which the database ID will be returned. database_name is of type sysname. When database_name is omitted, SQL Server PDW returns the ID of the current database.

Return Types

int

Examples

A. Return the ID of the current database

The following example returns the database ID of the current database.

Copy image Copy Code
SELECT DB_ID();

B. Return the ID of a named database.

The following example returns the database ID of the AdventureWorksDW2012 database.

Copy image Copy Code
SELECT DB_ID('AdventureWorksPDW2012');

See Also

Reference

Functions (SQL Server PDW)


DB_NAME (SQL Server PDW)

[image]

DB_NAME (SQL Server PDW)
See Also Send Feedback

Returns the name of a database in SQL Server PDW.

For more information, see the DB_NAME (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DB_NAME  ( [database_id] )

Arguments

database_id

Is the ID, specified as an integer, for the database. When database_id is omitted, SQL Server PDW returns the name of the current database.

Return Types

nvarchar(128)

Examples

A. Return the current database name

Copy image Copy Code
SELECT DB_NAME() AS [Current Database];

B. Return the name of a database by using the database ID

The following example returns the database name and database_id for each database.

Copy image Copy Code
SELECT DB_NAME(database_id) AS [Database], database_id
FROM sys.databases;

See Also

Reference

Functions (SQL Server PDW)


DEGREES (SQL Server PDW)

[image]

DEGREES (SQL Server PDW)
See Also Send Feedback

Returns the corresponding angle in degrees for an angle specified in radians.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DEGREES ( numeric_expression )

Arguments

numeric_expression

Is an expression of the exact numeric or approximate numeric data type category, except for the bit data type.

Return Code Values

Returns the same type as numeric_expression.

Examples

The following example returns the number of degrees in an angle of PI/2 radians.

Copy image Copy Code
SELECT 'The number of degrees in PI/2 radians is: ' +
CONVERT(varchar, DEGREES((PI()/2)));
GO

Here is the result set.

Copy image Copy Code
The number of degrees in PI/2 radians is 90

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


DENSE_RANK (SQL Server PDW)

[image]

DENSE_RANK (SQL Server PDW)
See Also Send Feedback

Returns the rank of rows that is contained in each partition of a result set, without any gaps in the ranking in SQL Server PDW. The rank of a row is one plus the number of distinct ranks that come before the row in question. Use this analytic function to assign a ranking value to a set of rows according to some criterion such as an increasing numeric value.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DENSE_RANK ( ) OVER ( [ [ partition_by_clause ] order_by_clause ] )

Arguments

OVER

Determines the partitioning and ordering of the rowset before the analytic function is applied.

partition_by_clause

Divides the result set produced by the FROM (SQL Server PDW) clause into partitions to which the DENSE_RANK function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

order_by_clause

Determines the order in which the DENSE_RANK values are applied to the rows in a partition. An integer cannot represent a column in order_by_clause. For more information about order_by_clause, see ORDER BY (SQL Server PDW).

Return Types

bigint

General Remarks

If two or more rows tie for a rank in the same partition, each tied rows receives the same rank. For example, if the two top salespeople have the same SalesYTD value, they are both ranked as number one. The salesperson with the next highest SalesYTD is ranked as number two. This is one more than the number of distinct rows that come before this row. Therefore, the numbers returned by the DENSE_RANK function do not have gaps and always have consecutive ranks. This behavior differs from the RANK function.

The sort order used for the whole query determines the order in which the rows appear in a result. This implies that a row ranked number one does not have to be the first row in the partition.

Examples

The following example ranks the sales representatives in each sales territory according to their total sales. The rowset is partitioned by SalesTerritoryGroup and sorted by SalesAmountQuota.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT LastName, SUM(SalesAmountQuota) AS TotalSales, SalesTerritoryGroup,
    DENSE_RANK() OVER (PARTITION BY SalesTerritoryGroup ORDER BY SUM(SalesAmountQuota) DESC ) AS RankResult
FROM dbo.DimEmployee AS e
INNER JOIN dbo.FactSalesQuota AS sq ON e.EmployeeKey = sq.EmployeeKey
INNER JOIN dbo.DimSalesTerritory AS st ON e.SalesTerritoryKey = st.SalesTerritoryKey
WHERE SalesPersonFlag = 1 AND SalesTerritoryGroup != N'NA'
GROUP BY LastName,SalesTerritoryGroup;

Here is the result set.

LastName TotalSales SalesTerritoryGroup RankResult

---------------- ------------- ------------------- --------

Pak 10514000.0000 Europe 1

Varkey Chudukatil 5557000.0000 Europe 2

Valdez 2287000.0000 Europe 3

Carson 12198000.0000 North America 1

Mitchell 11786000.0000 North America 2

Blythe 11162000.0000 North America 3

Reiter 8541000.0000 North America 4

Ito 7804000.0000 North America 5

Saraiva 7098000.0000 North America 6

Vargas 4365000.0000 North America 7

Campbell 4025000.0000 North America 8

Ansman-Wolfe 3551000.0000 North America 9

Mensa-Annan 2753000.0000 North America 10

Tsoflias 1687000.0000 Pacific 1

See Also

Reference

RANK (SQL Server PDW)
ROW_NUMBER (SQL Server PDW)
Functions (SQL Server PDW)


DIFFERENCE (SQL Server PDW)

[image]

DIFFERENCE (SQL Server PDW)
See Also Send Feedback

Returns an integer value that indicates the difference between the SOUNDEX values of two character expressions.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DIFFERENCE ( character_expression , character_expression )

Arguments

character_expression

Is an alphanumeric expression of character data. character_expression can be a constant, variable, or column.

Return Types

int

Remarks

The integer returned is the number of characters in the SOUNDEX values that are the same. The return value ranges from 0 through 4: 0 indicates weak or no similarity, and 4 indicates strong similarity or the same values.

DIFFERENCE and SOUNDEX are collation sensitive.

Examples

In the first part of the following example, the SOUNDEX values of two very similar strings are compared. For a Latin1_General collation DIFFERENCE returns a value of 4. In the second part of the following example, the SOUNDEX values for two very different strings are compared, and for a Latin1_General collation DIFFERENCE returns a value of 0.

Copy image Copy Code
-- Returns a DIFFERENCE value of 4, the least possible difference.
SELECT SOUNDEX('Green'), SOUNDEX('Greene'), DIFFERENCE('Green','Greene');
GO
-- Returns a DIFFERENCE value of 0, the highest possible difference.
SELECT SOUNDEX('Blotchet-Halls'), SOUNDEX('Greene'), DIFFERENCE('Blotchet-Halls', 'Greene');
GO

Here is the result set.

Copy image Copy Code
----- ----- -----------
G650  G650  4

(1 row(s) affected)

----- ----- -----------
B432  G650  0

(1 row(s) affected)

See Also

Reference

SOUNDEX (SQL Server PDW)
Functions (SQL Server PDW)


EOMONTH (SQL Server PDW)

[image]

EOMONTH (SQL Server PDW)
See Also Send Feedback

Returns the last day of the month that contains the specified date, with an optional offset.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

EOMONTH ( start_date [, month_to_add ] )

Arguments

start_date

Date expression specifying the date for which to return the last day of the month.

month_to_add

Optional integer expression specifying the number of months to add to start_date.

If this argument is specified, then EOMONTH adds the specified number of months to start_date, and then returns the last day of the month for the resulting date. If this addition overflows the valid range of dates, then an error is raised.

Return Type

date

Examples

A. EOMONTH with explicit datetime type

Copy image Copy Code
DECLARE @date DATETIME = '12/1/2011';
SELECT EOMONTH ( @date ) AS Result;
GO

Here is the result set.

Copy image Copy Code
Result
------------
2011-12-31

(1 row(s) affected)

B. EOMONTH with string parameter and implicit conversion

Copy image Copy Code
DECLARE @date VARCHAR(255) = '12/1/2011';
SELECT EOMONTH ( @date ) AS Result;
GO

Here is the result set.

Copy image Copy Code
Result
------------
2011-12-31

(1 row(s) affected)

C. EOMONTH with and without the month_to_add parameter

Transact-SQL Copy image Copy Code
DECLARE @date DATETIME = GETDATE();
SELECT EOMONTH ( @date ) AS 'This Month';
SELECT EOMONTH ( @date, 1 ) AS 'Next Month';
SELECT EOMONTH ( @date, -1 ) AS 'Last Month';
GO

Here is the result set.

Copy image Copy Code
This Month
-----------------------
2011-12-31

(1 row(s) affected)

Next Month
-----------------------
2012-01-31

(1 row(s) affected)

Last Month
-----------------------
2011-11-30

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


ERROR_MESSAGE (SQL Server PDW)

[image]

ERROR_MESSAGE (SQL Server PDW)
See Also Send Feedback

Returns the message text of the error that caused the CATCH block of a TRY…CATCH construct to be run.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ERROR_MESSAGE ( )

Return Types

nvarchar(4000)

Return Value

When called in a CATCH block, returns the complete text of the error message that caused the CATCH block to be run. The text includes the values supplied for any substitutable parameters, such as lengths, object names, or times.

Returns NULL if called outside the scope of a CATCH block.

Remarks

ERROR_MESSAGE may be called anywhere within the scope of a CATCH block.

ERROR_MESSAGE returns the error message regardless of how many times it is run, or where it is run within the scope of the CATCH block. This is in contrast to functions like @@ERROR, which only returns an error number in the statement immediately after the one that causes an error, or the first statement of a CATCH block.

In nested CATCH blocks, ERROR_MESSAGE returns the error message specific to the scope of the CATCH block in which it is referenced. For example, the CATCH block of an outer TRY...CATCH construct could have a nested TRY...CATCH construct. Within the nested CATCH block, ERROR_MESSAGE returns the message from the error that invoked the nested CATCH block. If ERROR_MESSAGE is run in the outer CATCH block, it returns the message from the error that invoked that CATCH block.

Examples

A. Using ERROR_MESSAGE in a CATCH block

The following code example shows a SELECT statement that generates a divide-by-zero error. The message of the error is returned.

Copy image Copy Code
BEGIN TRY
    -- Generate a divide-by-zero error.
    SELECT 1/0;
END TRY
BEGIN CATCH
    SELECT ERROR_MESSAGE() AS ErrorMessage;
END CATCH;
GO

B. Using ERROR_MESSAGE in a CATCH block with other error-handling tools

The following code example shows a SELECT statement that generates a divide-by-zero error. Along with the error message, information that relates to the error is returned.

Copy image Copy Code
BEGIN TRY
    -- Generate a divide-by-zero error.
    SELECT 1/0;
END TRY
BEGIN CATCH
    SELECT
        ERROR_NUMBER() AS ErrorNumber
        ,ERROR_SEVERITY() AS ErrorSeverity
        ,ERROR_STATE() AS ErrorState
        ,ERROR_PROCEDURE() AS ErrorProcedure
        ,ERROR_MESSAGE() AS ErrorMessage;
END CATCH;
GO

See Also

Reference

TRY...CATCH (SQL Server PDW)
@@ERROR (SQL Server PDW)
ERROR_NUMBER (SQL Server PDW)
ERROR_PROCEDURE (SQL Server PDW)
ERROR_SEVERITY (SQL Server PDW)
ERROR_STATE (SQL Server PDW)
THROW (SQL Server PDW)


ERROR_NUMBER (SQL Server PDW)

[image]

ERROR_NUMBER (SQL Server PDW)
See Also Send Feedback

Returns the error number of the error that caused the CATCH block of a TRY…CATCH construct to be run.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ERROR_NUMBER ( )

Return Types

int

Return Value

When called in a CATCH block, returns the error number of the error message that caused the CATCH block to be run.

Returns NULL if called outside the scope of a CATCH block.

Remarks

This function may be called anywhere within the scope of a CATCH block.

ERROR_NUMBER returns the error number regardless of how many times it is run, or where it is run within the scope of the CATCH block. This is in contrast to @@ERROR, which only returns the error number in the statement immediately after the one that causes an error, or the first statement of a CATCH block.

In nested CATCH blocks, ERROR_NUMBER returns the error number specific to the scope of the CATCH block in which it is referenced. For example, the CATCH block of an outer TRY...CATCH construct could have a nested TRY...CATCH construct. Within the nested CATCH block, ERROR_NUMBER returns the number from the error that invoked the nested CATCH block. If ERROR_NUMBER is run in the outer CATCH block, it returns the number from the error that invoked that CATCH block.

Examples

A. Using ERROR_NUMBER in a CATCH block

The following code example shows a SELECT statement that generates a divide-by-zero error. The number of the error is returned.

Copy image Copy Code
BEGIN TRY
    -- Generate a divide-by-zero error.
    SELECT 1/0;
END TRY
BEGIN CATCH
    SELECT ERROR_NUMBER() AS ErrorNumber;
END CATCH;
GO

B. Using ERROR_NUMBER in a CATCH block with other error-handling tools

The following code example shows a SELECT statement that generates a divide-by-zero error. Along with the error number, information that relates to the error is returned.

Copy image Copy Code
BEGIN TRY
    -- Generate a divide-by-zero error.
    SELECT 1/0;
END TRY
BEGIN CATCH
    SELECT
        ERROR_NUMBER() AS ErrorNumber,
        ERROR_SEVERITY() AS ErrorSeverity,
        ERROR_STATE() AS ErrorState,
        ERROR_PROCEDURE() AS ErrorProcedure,
        ERROR_MESSAGE() AS ErrorMessage;
END CATCH;
GO

See Also

Reference

TRY...CATCH (SQL Server PDW)
@@ERROR (SQL Server PDW)
ERROR_MESSAGE (SQL Server PDW)
ERROR_PROCEDURE (SQL Server PDW)
ERROR_SEVERITY (SQL Server PDW)
ERROR_STATE (SQL Server PDW)
THROW (SQL Server PDW)


ERROR_PROCEDURE (SQL Server PDW)

[image]

ERROR_PROCEDURE (SQL Server PDW)
See Also Send Feedback

Returns the name of the stored procedure where an error occurred that caused the CATCH block of a TRY…CATCH construct to be run.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ERROR_PROCEDURE ( )

Return Types

nvarchar(128)

Return Value

When called in a CATCH block, returns the stored procedure name where the error occurred.

Returns NULL if the error did not occur within a stored procedure.

Returns NULL if called outside the scope of a CATCH block.

Remarks

ERROR_PROCEDURE may be called anywhere within the scope of a CATCH block.

ERROR_PROCEDURE returns the name of the stored procedure where the error occurred, regardless of the number of times it is called or where it is called within the scope of the CATCH block. This contrasts with functions, such as @@ERROR, which return the error number in the statement immediately following the one that caused the error or in the first statement of the CATCH block.

In nested CATCH blocks, ERROR_PROCEDURE returns the name of the stored procedure specific to the scope of the CATCH block in which it is referenced. For example, the CATCH block of a TRY…CATCH construct could have a nested TRY…CATCH. Within the nested CATCH block, ERROR_PROCEDURE returns the name of the stored procedure where the error occurred that invoked the nested CATCH block. If ERROR_PROCEDURE is run in the outer CATCH block, it returns the name of the stored procedure where the error occurred that invoked that CATCH block.

Examples

A. Using ERROR_PROCEDURE in a CATCH block

The following code example shows a stored procedure that generates a divide-by-zero error. ERROR_PROCEDURE returns the name of the stored procedure in which the error occurred.

Copy image Copy Code
-- Verify that the stored procedure does not already exist.
IF OBJECT_ID ( 'usp_ExampleProc', 'P' ) IS NOT NULL
    DROP PROCEDURE usp_ExampleProc;
GO

-- Create a stored procedure that
-- generates a divide-by-zero error.
CREATE PROCEDURE usp_ExampleProc
AS
    SELECT 1/0;
GO

BEGIN TRY
    -- Execute the stored procedure inside the TRY block.
    EXECUTE usp_ExampleProc;
END TRY
BEGIN CATCH
    SELECT ERROR_PROCEDURE() AS ErrorProcedure;
END CATCH;
GO

B. Using ERROR_PROCEDURE in a CATCH block with other error-handling tools

The following code example shows a stored procedure that generates a divide-by-zero error. Along with the name of the stored procedure in which the error occurred, information that relates to the error is returned.

Copy image Copy Code
-- Verify that the stored procedure does not already exist.
IF OBJECT_ID ( 'usp_ExampleProc', 'P' ) IS NOT NULL
    DROP PROCEDURE usp_ExampleProc;
GO

-- Create a stored procedure that
-- generates a divide-by-zero error.
CREATE PROCEDURE usp_ExampleProc
AS
    SELECT 1/0;
GO

BEGIN TRY
    -- Execute the stored procedure inside the TRY block.
    EXECUTE usp_ExampleProc;
END TRY
BEGIN CATCH
    SELECT
        ERROR_NUMBER() AS ErrorNumber,
        ERROR_SEVERITY() AS ErrorSeverity,
        ERROR_STATE() AS ErrorState,
        ERROR_PROCEDURE() AS ErrorProcedure,
        ERROR_MESSAGE() AS ErrorMessage;
        END CATCH;
GO

See Also

Reference

TRY...CATCH (SQL Server PDW)
@@ERROR (SQL Server PDW)
ERROR_MESSAGE (SQL Server PDW)
ERROR_NUMBER (SQL Server PDW)
ERROR_SEVERITY (SQL Server PDW)
ERROR_STATE (SQL Server PDW)
THROW (SQL Server PDW)


ERROR_SEVERITY (SQL Server PDW)

[image]

ERROR_SEVERITY (SQL Server PDW)
See Also Send Feedback

Returns the severity of the error that caused the CATCH block of a TRY…CATCH construct to be run.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ERROR_SEVERITY ( )

Return Types

int

Return Value

When called in a CATCH block, returns the severity of the error message that caused the CATCH block to be run.

Returns NULL if called outside the scope of a CATCH block.

Remarks

ERROR_SEVERITY may be called anywhere within the scope of a CATCH block.

ERROR_SEVERITY returns the error severity regardless of how many times it is run, or where it is run within the scope of the CATCH block. This is in contrast to functions like @@ERROR, which only returns the error number in the statement immediately after the one that causes an error, or in the first statement of a CATCH block.

In nested CATCH blocks, ERROR_SEVERITY returns the error severity specific to the scope of the CATCH block in which it is referenced. For example, the CATCH block of an outer TRY...CATCH construct could have a nested TRY...CATCH construct. Within the nested CATCH block, ERROR_SEVERITY returns the severity from the error that invoked the nested CATCH block. If ERROR_SEVERITY is run in the outer CATCH block, it returns the severity from the error that invoked that CATCH block.

Examples

A. Using ERROR_SEVERITY in a CATCH block

The following example shows a SELECT statement that generates a divide-by-zero error. The severity of the error is returned.

Copy image Copy Code
BEGIN TRY
    -- Generate a divide-by-zero error.
    SELECT 1/0;
END TRY
BEGIN CATCH
    SELECT ERROR_SEVERITY() AS ErrorSeverity;
END CATCH;
GO

B. Using ERROR_SEVERITY in a CATCH block with other error-handling tools

The following example shows a SELECT statement that generates a divide by zero error. Along with the severity, information that relates to the error is returned.

Copy image Copy Code
BEGIN TRY
    -- Generate a divide-by-zero error.
    SELECT 1/0;
END TRY
BEGIN CATCH
    SELECT
        ERROR_NUMBER() AS ErrorNumber,
        ERROR_SEVERITY() AS ErrorSeverity,
        ERROR_STATE() AS ErrorState,
        ERROR_PROCEDURE() AS ErrorProcedure,
        ERROR_MESSAGE() AS ErrorMessage;
END CATCH;
GO

See Also

Reference

TRY...CATCH (SQL Server PDW)
@@ERROR (SQL Server PDW)
ERROR_MESSAGE (SQL Server PDW)
ERROR_NUMBER (SQL Server PDW)
ERROR_PROCEDURE (SQL Server PDW)
ERROR_STATE (SQL Server PDW)
THROW (SQL Server PDW)


ERROR_STATE (SQL Server PDW)

[image]

ERROR_STATE (SQL Server PDW)
See Also Send Feedback

Returns the state number of the error that caused the CATCH block of a TRY…CATCH construct to be run.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ERROR_STATE ( )

Return Types

int

Return Value

When called in a CATCH block, returns the state number of the error message that caused the CATCH block to be run.

Returns NULL if called outside the scope of a CATCH block.

Remarks

Some error messages can be raised at multiple points in the code. For example, an "1105" error can be raised for several different conditions. Each specific condition that raises the error assigns a unique state code.

When viewing databases of known issues, such as the Microsoft Knowledge Base, you can use the state number to determine if the recorded issue might be the same as the error you have encountered. For example, if a Knowledge Base article discusses an 1105 error message with a state of 2, and the 1105 error message you received had a state of 3, your error probably had a different cause than the one reported in the article.

A support engineer can also use the state code from an error to find the location in the source code where that error is being raised, which may provide additional ideas on how to diagnose the problem.

ERROR_STATE may be called anywhere within the scope of a CATCH block.

ERROR_STATE returns the error state regardless of how many times it is run, or where it is run within the scope of the CATCH block. This is in contrast to functions like @@ERROR, which only returns the error number in the statement immediately after the one that causes an error, or in the first statement of a CATCH block.

In nested CATCH blocks, ERROR_STATE returns the error state specific to the scope of the CATCH block in which it is referenced. For example, the CATCH block of an outer TRY...CATCH construct could have a nested TRY...CATCH construct. Within the nested CATCH block, ERROR_STATE returns the state from the error that invoked the nested CATCH block. If ERROR_STATE is run in the outer CATCH block, it returns the state from the error that invoked that CATCH block.

Examples

A. Using ERROR_STATE in a CATCH block

The following example shows a SELECT statement that generates a divide-by-zero error. The state of the error is returned.

Copy image Copy Code
BEGIN TRY
    -- Generate a divide by zero error
    SELECT 1/0;
END TRY
BEGIN CATCH
    SELECT ERROR_STATE() AS ErrorState;
END CATCH;
GO

B. Using ERROR_STATE in a CATCH block with other error-handling tools

The following example shows a SELECT statement that generates a divide-by-zero error. Along with the error state, information that relates to the error is returned.

Copy image Copy Code
BEGIN TRY
    -- Generate a divide-by-zero error.
    SELECT 1/0;
END TRY
BEGIN CATCH
    SELECT
        ERROR_NUMBER() AS ErrorNumber,
        ERROR_SEVERITY() AS ErrorSeverity,
        ERROR_STATE() AS ErrorState,
        ERROR_PROCEDURE() AS ErrorProcedure,
        ERROR_MESSAGE() AS ErrorMessage;
END CATCH;
GO

See Also

Reference

TRY...CATCH (SQL Server PDW)
@@ERROR (SQL Server PDW)
ERROR_MESSAGE (SQL Server PDW)
ERROR_NUMBER (SQL Server PDW)
ERROR_PROCEDURE (SQL Server PDW)
ERROR_SEVERITY (SQL Server PDW)
THROW (SQL Server PDW)


EXP (SQL Server PDW)

[image]

EXP (SQL Server PDW)
See Also Send Feedback

Returns the exponential value of the specified float expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

EXP ( float_expression )

Arguments

float_expression

Is an expression of type float or of a type that can be implicitly converted to float.

Return Types

float

Remarks

The constant e (2.718281…), is the base of natural logarithms.

The exponential value of a number is the constant e raised to the power of the number. For example EXP(1.0) = e^1.0 = 2.71828182845905 and EXP(10) = e^10 = 22026.4657948067.

The exponential value is the inverse function of the natural logarithm. The exponent of the natural logarithm of a number is the number itself: EXP (LOG (n)) = n. And the natural logarithm of the exponent of a number is the number itself: LOG (EXP (n)) = n.

Examples

A. Finding the exponent of a number

The following example returns the exponential value of the specified value (10).

Copy image Copy Code
SELECT EXP(10);

Here is the result set.

Copy image Copy Code
----------
22026.4657948067

B. Finding exponential values and natural logarithms

The following example returns the exponential value of the natural logarithm of 20 and the natural logarithm of the exponential of 20. Because these functions are inverse functions of one another, the return value in both cases is 20.

Copy image Copy Code
SELECT EXP( LOG(20)), LOG( EXP(20));

Here is the result set.

Copy image Copy Code
-------------- -----------------
20                  20

See Also

Reference

Expressions (SQL Server PDW)
POWER (SQL Server PDW)
LOG (SQL Server PDW)
* (Multiply) (SQL Server PDW)


FLOOR (SQL Server PDW)

[image]

FLOOR (SQL Server PDW)
See Also Send Feedback

Returns the largest integer smaller than the specified numeric expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

FLOOR ( numeric_expression )

Arguments

numeric_expression

An expression of the exact numeric or approximate numeric data type categories, except for the bit data type.

Return Types

Returns the same type as numeric_expression.

Examples

The following example shows positive numeric, negative numeric, and values with the FLOOR function.

Copy image Copy Code
SELECT FLOOR(123.45), FLOOR(-123.45), FLOOR($123.45);

The result is the integer part of the calculated value in the same data type as numeric_expression.

----- --------- -----------

123 -124 123

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


GETDATE (SQL Server PDW)

[image]

GETDATE (SQL Server PDW)
See Also Send Feedback

Returns the current database system timestamp as a datetime value without the database time zone offset. This value is derived from the operating system of the SQL Server PDW appliance.

Note Note

SYSDATETIME has more fractional seconds precision than GETDATE. SYSDATETIME can be assigned to a variable of any of the date and time types.

For an overview of all standard Transact-SQL date and time data types and functions, see Date and Time Data Types and Functions.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

GETDATE ( )

Return Type

datetime

Remarks

Transact-SQL statements can refer to GETDATE anywhere they can refer to a datetime expression.

GETDATE is a nondeterministic function. Views and expressions that reference this function in a column cannot be indexed.

Examples

The following examples use the three SQL Server system functions that return current date and time to return the date, time, or both. The values are returned in series; therefore, their fractional seconds might be different.

A. Getting the current system date and time

Copy image Copy Code
SELECT SYSDATETIME()
    ,CURRENT_TIMESTAMP
    ,GETDATE();

B. Getting the current system date

Copy image Copy Code
SELECT CONVERT (date, SYSDATETIME())
    ,CONVERT (date, CURRENT_TIMESTAMP)
    ,CONVERT (date, GETDATE());

C. Getting the current system time

Copy image Copy Code
SELECT CONVERT (time, SYSDATETIME())
    ,CONVERT (time, CURRENT_TIMESTAMP)
    ,CONVERT (time, GETDATE());

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
DATEPART (SQL Server PDW)
DATEDIFF (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


GETUTCDATE (SQL Server PDW)

[image]

GETUTCDATE (SQL Server PDW)
See Also Send Feedback

Returns the current database system timestamp as a datetime value. The database time zone offset is not included. This value represents the current UTC time (Coordinated Universal Time). This value is derived from the operating system.

Note Note

SYSDATETIME and SYSUTCDATETIME have more fractional seconds precision than GETDATE and GETUTCDATE. SYSDATETIMEOFFSET includes the system time zone offset. SYSDATETIME, SYSUTCDATETIME, and SYSDATETIMEOFFSET can be assigned to a variable of any of the date and time types.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

GETUTCDATE()

Return Types

datetime

Remarks

Transact-SQL statements can refer to GETUTCDATE anywhere they can refer to a datetime expression.

GETUTCDATE is a nondeterministic function. Views and expressions that reference this function in a column cannot be indexed.

Examples

The following examples use the six system functions that return current date and time to return the date, time or both. The values are returned in series; therefore, their fractional seconds might be different.

A. Getting the current system date and time

Copy image Copy Code
SELECT 'SYSDATETIME()      ', SYSDATETIME();
SELECT 'SYSDATETIMEOFFSET()', SYSDATETIMEOFFSET();
SELECT 'SYSUTCDATETIME()   ', SYSUTCDATETIME();
SELECT 'CURRENT_TIMESTAMP  ', CURRENT_TIMESTAMP;
SELECT 'GETDATE()          ', GETDATE();
SELECT 'GETUTCDATE()       ', GETUTCDATE();
/* Returned:
SYSDATETIME()            2007-05-03 18:34:11.9351421
SYSDATETIMEOFFSET()      2007-05-03 18:34:11.9351421 -07:00
SYSUTCDATETIME()         2007-05-04 01:34:11.9351421
CURRENT_TIMESTAMP        2007-05-03 18:34:11.933
GETDATE()                2007-05-03 18:34:11.933
GETUTCDATE()             2007-05-04 01:34:11.933
*/

B. Getting the current system date

Copy image Copy Code
SELECT 'SYSDATETIME()      ', CONVERT (date, SYSDATETIME());
SELECT 'SYSDATETIMEOFFSET()', CONVERT (date, SYSDATETIMEOFFSET());
SELECT 'SYSUTCDATETIME()   ', CONVERT (date, SYSUTCDATETIME());
SELECT 'CURRENT_TIMESTAMP  ', CONVERT (date, CURRENT_TIMESTAMP);
SELECT 'GETDATE()          ', CONVERT (date, GETDATE());
SELECT 'GETUTCDATE()       ', CONVERT (date, GETUTCDATE());

/* Returned:
SYSDATETIME()            2007-05-03
SYSDATETIMEOFFSET()      2007-05-03
SYSUTCDATETIME()         2007-05-04
CURRENT_TIMESTAMP        2007-05-03
GETDATE()                2007-05-03
GETUTCDATE()             2007-05-04
*/

C. Getting the current system time

Copy image Copy Code
SELECT 'SYSDATETIME()      ', CONVERT (time, SYSDATETIME());
SELECT 'SYSDATETIMEOFFSET()', CONVERT (time, SYSDATETIMEOFFSET());
SELECT 'SYSUTCDATETIME()   ', CONVERT (time, SYSUTCDATETIME());
SELECT 'CURRENT_TIMESTAMP  ', CONVERT (time, CURRENT_TIMESTAMP);
SELECT 'GETDATE()          ', CONVERT (time, GETDATE());
SELECT 'GETUTCDATE()       ', CONVERT (time, GETUTCDATE());
/* Returned
SYSDATETIME()            18:25:01.6958841
SYSDATETIMEOFFSET()      18:25:01.6958841
SYSUTCDATETIME()         01:25:01.6958841
CURRENT_TIMESTAMP        18:25:01.6930000
GETDATE()                18:25:01.6930000
GETUTCDATE()             01:25:01.6930000
*/

See Also

Reference

Functions (SQL Server PDW)


HAS_DBACCESS (SQL Server PDW)

[image]

HAS_DBACCESS (SQL Server PDW)
Send Feedback

Returns information about whether the user has access to the specified database. For more information, see the HAS_DBACCESS (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

HAS_DBACCESS ( 'database_name' )

Arguments

'database_name'

The name of the database for which the user wants access information. database_name is sysname.

Return Types

int

Remarks

HAS_DBACCESS returns 1 if the user has access to the database, 0 if the user has no access to the database, and NULL if the database name is not valid.

HAS_DBACCESS returns 0 if the database is offline or suspect.

HAS_DBACCESS returns 0 if the database is in single-user mode and the database is in use by another user.

Permissions

Requires membership in the public role.

Examples

The following example tests whether current user has access to the AdventureWorksPDW2012 database.

Copy image Copy Code
SELECT HAS_DBACCESS('AdventureWorksPDW2012');
GO


HASHBYTES (SQL Server PDW)

[image]

HASHBYTES (SQL Server PDW)
See Also Send Feedback

Returns the MD2, MD4, MD5, SHA, SHA1, or SHA2 hash of its input.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

HASHBYTES ( '<algorithm>', { @input | 'input' } )

<algorithm>::= MD2 | MD4 | MD5 | SHA | SHA1 | SHA2_256 | SHA2_512

Arguments

'<algorithm>'

Identifies the hashing algorithm to be used to hash the input. This is a required argument with no default. The single quotation marks are required.

@input

Specifies a variable containing the data to be hashed. @input is varchar, nvarchar, or varbinary.

'input'

Specifies an expression that evaluates to a character or binary string to be hashed.

Allowed input values are limited to 8000 bytes. The output conforms to the algorithm standard: 128 bits (16 bytes) for MD2, MD4, and MD5; 160 bits (20 bytes) for SHA and SHA1; 256 bits (32 bytes) for SHA2_256, and 512 bits (64 bytes) for SHA2_512. For more information, see Choose an Encryption Algorithm.

Return Value

varbinary (maximum 8000 bytes)

Examples

Return the hash of a variable

The following example returns the SHA1 hash of the nvarchar data stored in variable @HashThis.

Copy image Copy Code
DECLARE @HashThis nvarchar(4000);
SET @HashThis = CONVERT(nvarchar(4000),'dslfdkjLK85kldhnv$n000#knf');
SELECT HASHBYTES('SHA1', @HashThis);

Return the hash of a table column

The following example returns the SHA1 hash of the values in column c1 in the table Test1.

Copy image Copy Code
CREATE TABLE dbo.Test1 (c1 nvarchar(50));
GO
INSERT dbo.Test1 VALUES ('This is a test.'), ('This is test 2.');
GO
SELECT HASHBYTES('SHA1', c1) FROM dbo.Test1;

Here is the result set.

Copy image Copy Code
-------------------------------------------
0x0E7AAB0B4FF0FD2DFB4F0233E2EE7A26CD08F173
0xF643A82F948DEFB922B12E50B950CEE130A934D6

(2 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


INDEXPROPERTY (SQL Server PDW)

[image]

INDEXPROPERTY (SQL Server PDW)
Send Feedback

Returns the named index or statistics property value of a specified table identification number, index or statistics name, and property name. For more information, see the INDEXPROPERTY (Transact-SQL) documentation on MSDN. Not all properties are valid in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

INDEXPROPERTY ( object_ID , index_or_statistics_name , property )

Arguments

object_ID

An expression that contains the object identification number of the table or indexed view for which to provide index property information. object_ID is int.

index_or_statistics_name

The name of the index or statistics for which to return property information. index_or_statistics_name is nvarchar(128).

property

The name of the database property to return. property is varchar(128), and can be one of these values.

Note Note

Unless noted otherwise, NULL is returned when property is not a valid property name, object_ID is not a valid object ID, object_ID is an unsupported object type for the specified property, or the caller does not have permission to view the object's metadata.

Property

Description

Value

IndexDepth

Depth of the index.

Number of index levels.

NULL = input is not valid.

IndexFillFactor

Fill factor value used when the index was created or last rebuilt.

Fill factor

IndexID

Index ID of the index on a specified table or indexed view.

Index ID

IsAutoStatistics

Statistics were generated by the AUTO_CREATE_STATISTICS option of ALTER DATABASE.

1 = True

0 = False or XML index.

IsClustered

Index is clustered.

1 = True

0 = False or XML index.

IsColumnstore

Index is an xVelocity memory optimized columnstore index

1 = True

0 = False

IsDisabled

Index is disabled.

1 = True

0 = False

NULL = Input is not valid.

IsFulltextKey

Index is the full-text key for a table.

1 = True

0 = False or XML index.

NULL = Input is not valid.

IsHypothetical

Index is hypothetical and cannot be used directly as a data access path. Hypothetical indexes hold column-level statistics and are maintained and used by Database Engine Tuning Advisor.

1 = True

0 = False or XML index

NULL = Input is not valid.

IsPadIndex

Index specifies space to leave open on each interior node.

1 = True

0 = False or XML index.

IsPageLockDisallowed

Page-locking value set by the ALLOW_PAGE_LOCKS option of ALTER INDEX.

1 = Page locking is disallowed.

0 = Page locking is allowed.

NULL = Input is not valid.

IsRowLockDisallowed

Row-locking value set by the ALLOW_ROW_LOCKS option of ALTER INDEX.

1 = Row locking is disallowed.

0 = Row locking is allowed.

NULL = Input is not valid.

IsStatistics

index_or_statistics_name is statistics created by the CREATE STATISTICS statement or by the AUTO_CREATE_STATISTICS option of ALTER DATABASE.

1 = True

0 = False

IsUnique

Index is unique.

1 = True

0 = False

Return Types

int

Exceptions

Returns NULL on error or if a caller does not have permission to view the object.

A user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as INDEXPROPERTY may return NULL if the user does not have any permission on the object.

Examples

The following example examines the properties of one of the indexes on the FactResellerSales table.

Copy image Copy Code
USE
    AdventureWorksPDW2012

GO
SELECT
INDEXPROPERTY(OBJECT_ID('dbo.FactResellerSales'),
    'ClusteredIndex_6d10fa223e5e4c1fbba087e29e16a7a2','IsClustered') AS [Is Clustered],
INDEXPROPERTY(OBJECT_ID('dbo.FactResellerSales'),
    'ClusteredIndex_6d10fa223e5e4c1fbba087e29e16a7a2','IsColumnstore') AS [Is Columnstore Index],
INDEXPROPERTY(OBJECT_ID('dbo.FactResellerSales'),
    'ClusteredIndex_6d10fa223e5e4c1fbba087e29e16a7a2','IndexFillFactor') AS [Fill Factor];
GO


ISDATE (SQL Server PDW)

[image]

ISDATE (SQL Server PDW)
See Also Send Feedback

Returns 1 if the expression is a valid date, time, or datetime value; otherwise, 0.

ISDATE returns 0 if the expression is a datetime2 value.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ISDATE ( expression )

Arguments

expression

Is a character string or expression that can be converted to a character string. The expression must be less than 4,000 characters. Date and time data types, except datetime and smalldatetime, are not allowed as the argument for ISDATE.

Return Type

int

Remarks

ISDATE is deterministic only if you use it with the CONVERT function, if the CONVERT style parameter is specified, and style is not equal to 0, 100, 9, or 109.

The return value of ISDATE depends on the language settings.

ISDATE expression Formats

For examples of valid formats for which ISDATE will return 1, see the section "Supported String Literal Formats for datetime" in the datetime and smalldatetime topics. For additional examples, also see the Input/Output column of the "Arguments" section of CAST and CONVERT.

The following table summarizes input expression formats that are not valid and that return 0 or an error.

ISDATE expression | ISDATE return value
NULL | 0
Values of data types listed in any data type category other than character strings, Unicode character strings, or date and time. | 0
Values of text, ntext, or image data types. | 0
Any value that has a seconds precision scale greater than 3, (.0000 through .0000000...n). ISDATE will return 0 if the expression is a datetime2 value, but will return 1 if the expression is a valid datetime value. | 0
Any value that mixes a valid date with an invalid value, for example 1995-10-1a. | 0

Examples

A. Using ISDATE to test for a valid datetime expression

The following example shows you how to use ISDATE to test whether a character string is a valid datetime.

Copy image Copy Code
IF ISDATE('2009-05-12 10:19:41.177') = 1
    SELECT 'VALID';
ELSE
    SELECT 'INVALID';

See Also

Reference

Functions (SQL Server PDW)
CAST and CONVERT (SQL Server PDW)


ISNULL (SQL Server PDW)

[image]

ISNULL (SQL Server PDW)
See Also Send Feedback

Replaces NULL with the specified replacement value in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ISNULL ( check_expression , replacement_value )

Arguments

check_expression

The expression to be checked for NULL. check_expression can be of any type.

replacement_value

The expression to be returned if check_expression is NULL. replacement_value must be of a type that is implicitly convertible to the type of check_expresssion.

Return Types

Returns the same type as check_expression.

Remarks

The value of check_expression is returned if it is not NULL; otherwise, replacement_value is returned after it is implicitly converted to the type of check_expression, if the types are different.

Do not use ISNULL to find NULL values. Use IS NULL instead.

Examples

A. Using ISNULL with AVG

The following example finds the average of the weight of all products in a sample table. It substitutes the value 50 for all NULL entries in the Weight column of the Product table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT AVG(ISNULL(Weight, 50))
FROM dbo.DimProduct;

Here is the result set.

Copy image Copy Code
--------------------------
52.88

B. Using ISNULL

The following example uses ISNULL to test for NULL values in the column MinPaymentAmount and display the value 0.00 for those rows.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT ResellerName,
       ISNULL(MinPaymentAmount,0) AS MinimumPayment
FROM dbo.DimReseller
ORDER BY ResellerName;

Here is a partial result set.

ResellerName MinimumPayment

------------------------- --------------

A Bicycle Association 0.0000

A Bike Store 0.0000

A Cycle Shop 0.0000

A Great Bicycle Company 0.0000

A Typical Bike Shop 200.0000

Acceptable Sales & Service 0.0000

C. Using IS NULL to test for NULL in a WHERE clause

The following example finds all products that have NULL in the Weight column. Note the space between IS and NULL.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EnglishProductName, Weight
FROM dbo.DimProduct
WHERE Weight IS NULL;

See Also

Reference

Expressions (SQL Server PDW)
NULLIF (SQL Server PDW)
WHERE (SQL Server PDW)


ISNUMERIC (SQL Server PDW)

[image]

ISNUMERIC (SQL Server PDW)
See Also Send Feedback

Determines whether an expression is a valid numeric type in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ISNUMERIC ( expression )

Arguments

expression

Is the expression to be evaluated.

Return Types

int

Remarks

ISNUMERIC returns 1 when the input expression evaluates to a valid numeric data type; otherwise it returns 0. Valid numeric data types include the following:

int | money
bigint | smallmoney
smallint | float
tinyint | real
decimal |
Note Note

ISNUMERIC returns 1 for some characters that are not numbers, such as plus (+), minus (-), and valid currency symbols such as the dollar sign ($).

Examples

The following example uses ISNUMERIC to return all the postal codes that are not numeric values.

Copy image Copy Code
USE master;
GO
SELECT name, isnumeric(name) AS IsNameANumber, database_id, isnumeric(database_id) AS IsIdANumber
FROM sys.databases;
GO

See Also

Reference

Expressions (SQL Server PDW)


LAG (SQL Server PDW)

[image]

LAG (SQL Server PDW)
See Also Send Feedback

Accesses data from a previous row in the same result set without the use of a self join in SQL Server PDW. Use this analytic function in a SELECT statement to compare values in the current row with values in a previous row.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

LAG (value_expression [,offset] [,default]) OVER ( [ [ partition_by_clause ] order_by_clause ] )

Arguments

value_expression

An expression of any type that returns a single (scalar) value. value_expression cannot be an analytic function.

offset

The number of rows back from the current row from which to obtain a value. If not specified, the default is 1. offset can be a column, a CASE expression, or other expression that evaluates to an integer. offset cannot be a negative value or an analytic function.

default

The value to assign to the column when no LAG value is available. For example, when the offset value is higher than the number of previous rows in the partition. If not specified, the default is NULL. default can be a column, CASE expression, or other expression, but it cannot be an analytic function. default must be type-compatible with value_expression.

OVER

Determines the partitioning and ordering of the rowset before the analytic function is applied.

partition_by_clause

Divides the result set produced by the FROM clause into partitions to which the LAG function is applied. For example, PARTITION BY FirstName. partition_by_clause can include one or more columns, a CASE expression, or another expression. NULL values in the partitioned columns are ignored. For more information about partition_by_clause, see OVER Clause (SQL Server PDW).

order_by_clause

Specifies the return order of the data values within each partition. For more information, see ORDER BY (SQL Server PDW). An integer cannot represent a column in order_by_clause.

Return Types

The data type of the specified value_expression.

Examples

The following example demonstrates the LAG function. The query uses the LAG function to return the difference in sales quotas for a specific employee over previous calendar quarters. Notice that because there is no lag value available for the first row, the default of zero (0) is returned.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT CalendarYear, CalendarQuarter, SalesAmountQuota AS SalesQuota,
       LAG(SalesAmountQuota,1,0) OVER (ORDER BY CalendarYear, CalendarQuarter) AS PrevQuota,
       SalesAmountQuota - LAG(SalesAmountQuota,1,0) OVER (ORDER BY CalendarYear, CalendarQuarter) AS Diff
FROM dbo.FactSalesQuota
WHERE EmployeeKey = 272 AND CalendarYear IN (2001, 2002)
ORDER BY CalendarYear, CalendarQuarter;

Here is the result set.

Year Quarter SalesQuota PrevQuota Diff

---- ------- ---------- --------- -------------

2001 3 28000.0000 0.0000 28000.0000

2001 4 7000.0000 28000.0000 -21000.0000

2001 1 91000.0000 7000.0000 84000.0000

2002 2 140000.0000 91000.0000 49000.0000

2002 3 7000.0000 140000.0000 -70000.0000

2002 4 154000.0000 7000.0000 84000.0000

See Also

Reference

LEAD (SQL Server PDW)
Functions (SQL Server PDW)
CASE (SQL Server PDW)


LEAD (SQL Server PDW)

[image]

LEAD (SQL Server PDW)
See Also Send Feedback

Accesses data from a subsequent row of data in the same result set without the use of a self join in SQL Server PDW. Use this analytic function in a SELECT statement to compare values in the current row with values in a following row.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

LEAD (value_expression [,offset] [,default]) OVER ( [ <partition_by_clause> ] <order_by_clause> )

Arguments

value_expression

An expression of any type that returns a single (scalar) value. Aggregate functions, analytic functions, and subqueries are not permitted.

Note Note

value_expression can only reference columns in the PARTITION BY clause and cannot reference columns listed in the FROM clause.

offset

The number of rows forward from the current row from which to obtain a value. If not specified, the default is 1. offset can be a column, a CASE expression, or other expression that evaluates to an integer. offset cannot be a negative value or an analytic function.

default

The value to assign to the column when no LEAD value is available. For example, when the offset value is higher than the number of previous rows in the partition. If not specified, the default is NULL. default can be a column, CASE expression, or other expression, but it cannot be an analytic function.

OVER

Determines the partitioning and ordering of the rowset before the analytic function is applied.

<partition_by_clause>

Divides the result set produced by the FROM clause into partitions to which the LEAD function is applied. For example, PARTITION BY FirstName. partition_by_clause can include one or more columns, a CASE expression, or another expression. NULL values in the partitioned columns are ignored. For more information about partition_by_clause, see OVER Clause (SQL Server PDW).

<order_by_clause>

Specifies the return order of the data values within each partition. For more information, see ORDER BY (SQL Server PDW). An integer cannot represent a column in order_by_clause.

Return Types

The data type of the specified value_expression.

Examples

The following example demonstrates the LEAD function. The query obtains the difference in sales quota values for a specified employee over subsequent calendar quarters. Notice that because there is no lead value available after the last row, the default of zero (0) is used.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT CalendarYear AS Year, CalendarQuarter AS Quarter, SalesAmountQuota AS SalesQuota,
       LEAD(SalesAmountQuota,1,0) OVER (ORDER BY CalendarYear, CalendarQuarter) AS NextQuota,
   SalesAmountQuota - LEAD(Sale sAmountQuota,1,0) OVER (ORDER BY CalendarYear, CalendarQuarter) AS Diff
FROM dbo.FactSalesQuota
WHERE EmployeeKey = 272 AND CalendarYear IN (2001,2002)
ORDER BY CalendarYear, CalendarQuarter;

Here is the result set.

Year Quarter SalesQuota NextQuota Diff

---- ------- ---------- --------- -------------

2001 3 28000.0000 7000.0000 21000.0000

2001 4 7000.0000 91000.0000 -84000.0000

2001 1 91000.0000 140000.0000 -49000.0000

2002 2 140000.0000 7000.0000 7000.0000

2002 3 7000.0000 154000.0000 84000.0000

2002 4 154000.0000 0.0000 154000.0000

See Also

Reference

LAG (SQL Server PDW)
Functions (SQL Server PDW)


LEFT (SQL Server PDW)

[image]

LEFT (SQL Server PDW)
See Also Send Feedback

Returns the left part of a character string with the specified number of characters in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

LEFT ( character_expression , integer_expression )

Arguments

character_expression

An expression of character or binary data. character_expression can be a constant, variable, or column. character_expression can be of any data type that can be implicitly converted to varchar or nvarchar. Otherwise, use the CAST function to explicitly convert character_expression.

integer_expression

A positive integer that specifies how many characters of the character_expression will be returned. If integer_expression is negative, an error is returned.

Return Types

Returns varchar when character_expression is a non-Unicode character data type.

Returns nvarchar when character_expression is a Unicode character data type.

Examples

A. Using LEFT with a column

The following example returns the five leftmost characters of each product name.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT LEFT(EnglishProductName, 5)
FROM dbo.DimProduct
ORDER BY ProductKey;

B. Using LEFT with a character string

The following example uses LEFT to return the two leftmost characters of the character string abcdefg.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT LEFT('abcdefg',2) FROM dbo.DimProduct;

Here is the result set.

Copy image Copy Code
--
ab

See Also

Reference

CAST and CONVERT (SQL Server PDW)
Functions (SQL Server PDW)
RIGHT (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


LEN (SQL Server PDW)

[image]

LEN (SQL Server PDW)
See Also Send Feedback

Returns the number of characters in the specified string expression, including leading but not trailing spaces in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

LEN ( string_expression )

Arguments

string_expression

The string expression to be evaluated. string_expression can be a constant or column of character data.

Return Types

int

Examples

The following example returns the number of characters in the column FirstName and the first and last names of employees located in Australia.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT LEN(FirstName) AS FNameLength, FirstName, LastName
FROM dbo.DimEmployee AS e
INNER JOIN dbo.DimGeography AS g
    ON e.SalesTerritoryKey = g.SalesTerritoryKey
WHERE EnglishCountryRegionName = 'Australia';

Here is the result set.

FNameLength FirstName LastName

----------- --------- ---------------

4 Lynn Tsoflias

See Also

Reference

Functions (SQL Server PDW)
LEFT (SQL Server PDW)
RIGHT (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


LOG (SQL Server PDW)

[image]

LOG (SQL Server PDW)
See Also Send Feedback

Returns the natural logarithm of the specified float expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

LOG ( float_expression )

Arguments

float_expression

Any valid float expression, or an expression of a type that can be implicitly converted to the float type. Otherwise, use the CAST function to convert float_expression.

Return Types

float

Remarks

The constant e (2.71828182845905…) is the base of natural logarithms.

The base of natural logarithms is the constant e (2.71828182845905…). LOG ( e ) = 1.0.

The natural logarithm of the exponential of a number is the number itself: LOG ( EXP ( n ) ) = n. And the exponential of the natural logarithm of a number is the number itself: EXP ( LOG ( n ) ) = n.

Examples

A. Calculating the logarithm for a number

The following example calculates the LOG for the specified float expression.

Copy image Copy Code
SELECT LOG(10);

Here is the result set.

----------------

2.30

B. Calculating the logarithm of the exponent of a number

The following example calculates the LOG for the exponent of a number.

Copy image Copy Code
SELECT LOG(EXP (10));

Here is the result set.

---------

10.00

See Also

Reference

Functions (SQL Server PDW)
EXP (SQL Server PDW)
LOG10 (SQL Server PDW)


LOG10 (SQL Server PDW)

[image]

LOG10 (SQL Server PDW)
See Also Send Feedback

Returns the base 10 logarithm of the specified float expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

LOG10 ( float_expression )

Arguments

float_expression

Is an expression of type float or of a type that can be implicitly converted to float. Otherwise, use CAST to convert the expression to the float type.

Return Types

float

Examples

The following example calculates the LOG10 of the specified variable.

Copy image Copy Code
SELECT LOG10(145.175642);

Here is the result set.

-------------------

2.16

See Also

Reference

Functions (SQL Server PDW)
LOG (SQL Server PDW)
EXP (SQL Server PDW)


LOWER (SQL Server PDW)

[image]

LOWER (SQL Server PDW)
See Also Send Feedback

Returns a character expression after converting uppercase character data to lowercase in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

LOWER ( character_expression )

Arguments

character_expression

An expression of character or binary data. character_expression can be a constant or column. character_expression must be of a data type that is implicitly convertible to varchar or nvarchar. Otherwise, use CAST to explicitly convert character_expression.

Return Types

varchar or nvarchar

Examples

The following example uses the LOWER function, the UPPER function, and nests the UPPER function inside the LOWER function in selecting product names that have prices between $11 and $20.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT LOWER(SUBSTRING(EnglishProductName, 1, 20)) AS Lower,
       UPPER(SUBSTRING(EnglishProductName, 1, 20)) AS Upper,
       LOWER(UPPER(SUBSTRING(EnglishProductName, 1, 20))) As LowerUpper
FROM dbo.DimProduct
WHERE ListPrice between 11.00 and 20.00;

Here is the result set.

Lower Upper LowerUpper

-------------------- --------------------- --------------------

minipump MINIPUMP minipump

taillights – battery TAILLIGHTS – BATTERY taillights - battery

See Also

Reference

Expressions (SQL Server PDW)
CAST and CONVERT (SQL Server PDW)
UPPER (SQL Server PDW)


LTRIM (SQL Server PDW)

[image]

LTRIM (SQL Server PDW)
See Also Send Feedback

Returns a character expression after it removes leading blanks in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

LTRIM ( character_expression )

Arguments

character_expression

An expression of character or binary data. character_expression can be a constant or column. character_expression must be of a data type that is implicitly convertible to varchar or nvarchar. Otherwise, use CAST to explicitly convert character_expression.

Return Type

varchar or nvarchar

Examples

The following example uses LTRIM to remove leading spaces from a character expression.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT LTRIM('     Five spaces are at the beginning of this string.') FROM dbo.DimCustomer;

Here is the result set.

---------------------------------------------------------------

Five spaces are at the beginning of this string.

See Also

Reference

CAST and CONVERT (SQL Server PDW)
Expressions (SQL Server PDW)
RTRIM (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


MAX (SQL Server PDW)

[image]

MAX (SQL Server PDW)
See Also Send Feedback

Returns the maximum value of an expression in SQL Server PDW. Use this function in the SELECT list or HAVING clause of a SELECT statement to return the single highest value for the specified expression in a table or partition.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Aggregation Function Syntax
MAX ( [ ALL | DISTINCT ] expression )
-- Aggregation Function Syntax
MAX ( expression ) OVER ( [ <partition_by_clause> ] [ <order_by_clause> ] )

Arguments

ALL

Applies the aggregate function to all values. ALL is the default.

DISTINCT

Considers only each unique value. DISTINCT is not meaningful with MAX and is available for ISO compatibility only.

expression

A constant, column name, or function, and any combination of arithmetic, bitwise, and string operators. MAX can be used with numeric, character, and date and time columns, except for the bit data type. Aggregate functions, subqueries, and analytic functions are not permitted.

For more information, see Expressions (SQL Server PDW).

OVER

Determines the partitioning and ordering of the rowset before the analytic function is applied.

< partition_by_clause >

Divides the result set produced by the FROM clause into partitions to which the MAX function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

< order_by_clause >

Determines the order in which the MAX calculation is applied to the rows in a partition. An integer cannot represent a column in order_by_clause. When the order_by_clause is absent, all rows within the same partition (as specified by the partition_by_clause) are assigned the same value for the MAX() OVER computation. When the order_by_clause is specified, the MAX() OVER computation gives you a cumulative maximum value within the partition. Within the a partition, rows with identical values for all elements of the order_by_clause return identical results for the MAX() OVER computation.

Return Types

Returns the same value type as expression.

General Remarks

MAX ignores any null values.

For character columns, MAX finds the highest value in the collating sequence.

Limitations and Restrictions

Unlike the MAX() aggregate function, the MAX() OVER analytic function does not support the use of the ALL or DISTINCT modifiers for the expression to compute the MAX against.

Examples

A. Using MAX

The following example uses the MAX aggregate function to return the price of the most expensive (maximum) product in a specified set of sales orders.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT MAX(UnitPrice)
FROM dbo.FactResellerSales
WHERE SalesOrderNumber IN (N'SO43659', N'SO43660', N'SO43664');

Here is the result set.

----------

2039.9940

B. Using MAX with OVER

The following examples use the MAX OVER() analytic function to return the price of the most expensive product in each sales order.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT MAX(UnitPrice) OVER(PARTITION BY SalesOrderNumber) AS MostExpensiveProduct,
       SalesOrderNumber
FROM dbo.FactResellerSales
WHERE SalesOrderNumber IN (N'SO43659', N'SO43660', N'SO43664')
ORDER BY SalesOrderNumber
;

Here is the result set.

MostExpensiveProduct SalesOrderNumber

--------------------- ----------------

2039.9940 SO43659

879.7940 SO43660

2039.9940 SO43664

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
MIN (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


MIN (SQL Server PDW)

[image]

MIN (SQL Server PDW)
See Also Send Feedback

Returns the minimum value in the expression in SQL Server PDW. Use this function in the select list or HAVING clause of a SELECT statement to return the single lowest value for the specified expression in a table or partition.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Aggregation Function Syntax
MIN ( [ ALL | DISTINCT ] expression )
-- Aggregation Function Syntax
MIN ( expression ) OVER ( [ <partition_by_clause> ] [ <order_by_clause> ] )

Arguments

ALL

Applies the aggregate function to all values. ALL is the default.

DISTINCT

Considers only each unique value. DISTINCT is not meaningful with MIN and is available for ISO compatibility only.

expression

A constant, column name, or function, and any combination of arithmetic, bitwise, and string operators. MIN can be used with numeric, character, and date and time columns, except for the bit data type. Aggregate functions, subqueries, and analytic functions are not permitted.

For more information, see Expressions (SQL Server PDW).

OVER

Determines the partitioning and ordering of the rowset before the analytic function is applied.

< partition_by_clause >

Divides the result set produced by the FROM clause into partitions to which the MIN function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

< order_by_clause >

Determines the order in which the MIN calculation is applied to the rows in a partition. An integer cannot represent a column in order_by_clause. When the order_by_clause is absent, all rows in the same partition (as specified by the partition_by_clause) are assigned the same value for the MIN() OVER computation. When the order_by_clause is specified, the MIN() OVER computation gives you a cumulative minimum value in the partition. Within the a partition, rows with identical values for all elements of the order_by_clause return identical results for the MIN() OVER computation.

Return Types

Returns the same value type as expression.

General Remarks

MIN ignores any null values.

For character columns, MIN finds the lowest value in the collating sequence.

Limitations and Restrictions

Unlike the MIN() aggregate function, the MIN() OVER analytic function does not support the use of the ALL or DISTINCT modifiers for the expression to compute the MIN against.

Examples

A. Using MIN

The following example uses the MIN aggregate function to return the price of the least expensive (minimum) product in a specified set of sales orders.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT MIN(UnitPrice)
FROM dbo.FactResellerSales
WHERE SalesOrderNumber IN (N'SO43659', N'SO43660', N'SO43664');

Here is the result set.

------

5.1865

B. Using MIN with OVER

The following examples use the MIN OVER() analytic function to return the price of the least expensive product in each sales order. The result set is partitioned by the SalesOrderID column.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT MIN(UnitPrice) OVER(PARTITION BY SalesOrderNumber) AS LeastExpensiveProduct,
       SalesOrderNumber
FROM dbo.FactResellerSales
WHERE SalesOrderNumber IN (N'SO43659', N'SO43660', N'SO43664')
ORDER BY SalesOrderNumber;

Here is the result set.

LeastExpensiveProduct SalesOrderID

--------------------- ----------

5.1865 SO43659

419.4589 SO43660

28.8404 SO43664

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)
MIN (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


MONTH (SQL Server PDW)

[image]

MONTH (SQL Server PDW)
See Also Send Feedback

Returns an integer that represents the month of the specified date in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

MONTH ( date )

Arguments

date

An expression that can be resolved to a time, date, smalldatetime, datetime, datetime2, or datetimeoffset value. The date argument can be an expression, column expression, or string literal.

Return Type

int

Return Value

MONTH returns the same value as DATEPART (month, date).

If date contains only a time part, the return value is 1, the base month.

Examples

The following example returns 4. This is the number of the month.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP 1 MONTH('2007-04-30T01:01:01.1234')
FROM dbo.DimCustomer;

The following example returns 1900, 1, 1. The argument for date is the number 0. SQL Server interprets 0 as January 1, 1900.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP 1 YEAR(0), MONTH(0), DAY(0) FROM dbo.DimCustomer;

See Also

Reference

DAY (SQL Server PDW)
YEAR (SQL Server PDW)


NCHAR (SQL Server PDW)

[image]

NCHAR (SQL Server PDW)
See Also Send Feedback

Returns the Unicode character with the specified integer code, as defined by the Unicode standard.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

NCHAR ( integer_expression )

Arguments

integer_expression

When the collation of the database does not contain the supplementary character (SC) flag, this is a positive whole number from 0 through 65535 (0 through 0xFFFF). If a value outside this range is specified, NULL is returned. For more information about supplementary characters, see Collations (SQL Server PDW).

When the collation of the database supports the supplementary character (SC) flag, this is a positive whole number from 0 through 1114111 (0 through 0x10FFFF). If a value outside this range is specified, NULL is returned.

Return Types

nchar(1) when the default database collation does not support supplementary characters.

nvarchar(2) when the default database collation supports supplementary characters.

If the parameter integer_expression lies in the range 0 - 0xFFFF, only one character is returned. For higher values, NCHAR returns the corresponding surrogate pair. Do not construct a surrogate pair by using NCHAR(<High surrogate>) + NCHAR(<Low Surrogate>). Instead, use a database collation that supports supplementary characters and then specify the Unicode codepoint for the surrogate pair. The following example demonstrates both the old style method of constructing a surrogate pair and the preferred method of specifying the Unicode codepoint.

Copy image Copy Code
CREATE DATABASE test COLLATE Finnish_Swedish_100_CS_AS_SC;
DECLARE @d nvarchar(10) = N'𣅿';
-– Old style method.
SELECT NCHAR(0xD84C) + NCHAR(0xDD7F);

-- Preferred method.
SELECT NCHAR(143743);

-- Alternative preferred method.
SELECT NCHAR(UNICODE(@d));

Examples

A. Using NCHAR and UNICODE

The following example uses the UNICODE and NCHAR functions to print the UNICODE value and the NCHAR (Unicode character) of the second character of the København character string, and to print the actual second character, ø.

Copy image Copy Code
DECLARE @nstring nchar(8);
SET @nstring = N'København';
SELECT UNICODE(SUBSTRING(@nstring, 2, 1)),
   NCHAR(UNICODE(SUBSTRING(@nstring, 2, 1)));
GO

Here is the result set.

Copy image Copy Code
----------- -
248         ø
(1 row(s) affected)

B. Using SUBSTRING, UNICODE, CONVERT, and NCHAR

The following example uses the SUBSTRING, UNICODE, CONVERT, and NCHAR functions to print the character number, the Unicode character, and the UNICODE value of each character in the string København.

Copy image Copy Code
-- The @position variable holds the position of the character currently
-- being processed. The @nstring variable is the Unicode character
-- string to process.
DECLARE @position int, @nstring nchar(9);
-- Initialize the current position variable to the first character in
-- the string.
SET @position = 1;
-- Initialize the character string variable to the string to process.
-- Notice that there is an N before the start of the string. This
-- indicates that the data following the N is Unicode data.
SET @nstring = N'København';
-- Print the character number of the position of the string you are at,
-- the actual Unicode character you are processing, and the UNICODE
-- value for this particular character.
PRINT 'Character #' + ' ' + 'Unicode Character' + ' ' + 'UNICODE Value';
WHILE @position <= DATALENGTH(@nstring)
   BEGIN
   SELECT @position,
      NCHAR(UNICODE(SUBSTRING(@nstring, @position, 1))),
      CONVERT(NCHAR(17), SUBSTRING(@nstring, @position, 1)),
      UNICODE(SUBSTRING(@nstring, @position, 1))
   SELECT @position = @position + 1
   END;
GO

Here is the result set.

Copy image Copy Code
Character # Unicode Character UNICODE Value

----------- ---- ----------------- -----------
1           K    K                 75

(1 row(s) affected)

----------- ---- ----------------- -----------
2           ø    ø                 248

(1 row(s) affected)

----------- ---- ----------------- -----------
3           b    b                 98

(1 row(s) affected)

----------- ---- ----------------- -----------
4           e    e                 101

(1 row(s) affected)

----------- ---- ----------------- -----------
5           n    n                 110

(1 row(s) affected)

----------- ---- ----------------- -----------
6           h    h                 104

(1 row(s) affected)

----------- ---- ----------------- -----------
7           a    a                 97

(1 row(s) affected)

----------- ---- ----------------- -----------
8           v    v                 118

(1 row(s) affected)

----------- ---- ----------------- -----------
9           n    n                 110

(1 row(s) affected)

----------- ---- ----------------- -----------
10          NULL                   NULL

(1 row(s) affected)

----------- ---- ----------------- -----------
11          NULL                   NULL

(1 row(s) affected)

----------- ---- ----------------- -----------
12          NULL                   NULL

(1 row(s) affected)

----------- ---- ----------------- -----------
13          NULL                   NULL

(1 row(s) affected)

----------- ---- ----------------- -----------
14          NULL                   NULL

(1 row(s) affected)

----------- ---- ----------------- -----------
15          NULL                   NULL

(1 row(s) affected)

----------- ---- ----------------- -----------
16          NULL                   NULL

(1 row(s) affected)

----------- ---- ----------------- -----------
17          NULL                   NULL

(1 row(s) affected)

----------- ---- ----------------- -----------
18          NULL                   NULL

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


NTILE (SQL Server PDW)

[image]

NTILE (SQL Server PDW)
See Also Send Feedback

Distributes the rows in an ordered partition into a specified number of groups in SQL Server PDW. The groups are numbered, starting at one. Use this analytic function to return the number of the group to which each row belongs.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

NTILE (integer_expression) OVER ( [ <partition_by_clause> ] < order_by_clause > )

Arguments

integer_expression

Is a positive integer constant expression that specifies the number of groups into which each partition must be divided. integer_expression can be of type int, or bigint. integer_expression cannot contain column references or subqueries.

OVER

Determines the partitioning and ordering of the rowset before the analytic function is applied.

<partition_by_clause>

Divides the result set produced by the FROM clause into partitions to which the NTILE function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

< order_by_clause >

Determines the order in which the NTILE values are assigned to the rows in a partition. An integer cannot represent a column in <order_by_clause>. For more information about order_by_clause, see ORDER BY (SQL Server PDW).

Return Types

bigint

General Remarks

When the number of rows in a partition is not divisible by integer_expression, the function returns groups of two sizes that differ by one member. Larger groups come before smaller groups in the order specified by the OVER clause. For example, if the total number of rows is 53 and the number of groups is five, the first three groups will have 11 rows and the two remaining groups will have 10 rows each. If, on the other hand, the total number of rows is divisible by the number of groups, the rows will be evenly distributed among the groups. For example, if the total number of rows is 50, and there are five groups, each bucket will contain 10 rows.

Examples

A. Dividing rows into groups

The following example uses the NTILE function to divide a set of salespersons into four groups based on their assigned sales quota for the year 2003. Because the total number of rows is not divisible by the number of groups, the first group has five rows and the remaining groups have four rows each.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT e.LastName, NTILE(4) OVER(ORDER BY SUM(SalesAmountQuota) DESC) AS Quartile,
       CONVERT (varchar(13), SUM(SalesAmountQuota), 1) AS SalesQuota
FROM dbo.DimEmployee AS e
INNER JOIN dbo.FactSalesQuota AS sq
    ON e.EmployeeKey = sq.EmployeeKey
WHERE sq.CalendarYear = 2003
    AND SalesTerritoryKey IS NOT NULL AND SalesAmountQuota <> 0
GROUP BY e.LastName
ORDER BY Quartile, e.LastName;

Here is the result set.

LastName Quartile SalesYTD

----------------- -------- ------------

Blythe 1 4,716,000.00

Carson 1 4,350,000.00

Mitchell 1 4,682,000.00

Pak 1 5,142,000.00

Varkey Chudukatil 1 2,940,000.00

Ito 2 2,644,000.00

Reiter 2 2,768,000.00

Saraiva 2 2,293,000.00

Vargas 2 1,617,000.00

Ansman-Wolfe 3 1,183,000.00

Campbell 3 1,438,000.00

Mensa-Annan 3 1,481,000.00

Valdez 3 1,294,000.00

Abbas 4 172,000.00

Albert 4 651,000.00

Jiang 4 544,000.00

Tsoflias 4 867,000.00

B. Dividing the result set by using PARTITION BY

The following example adds the PARTITION BY argument to the code in example A. The rows are first partitioned by SalesTerritoryCountry and then divided into two groups within each SalesTerritoryCountry. Notice that the ORDER BY in the OVER clause orders the NTILE and the ORDER BY of the SELECT statement orders the result set.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT e.LastName, NTILE(2) OVER(PARTITION BY e.SalesTerritoryKey ORDER BY SUM(SalesAmountQuota) DESC) AS Quartile,
       CONVERT (varchar(13), SUM(SalesAmountQuota), 1) AS SalesQuota
   ,st.SalesTerritoryCountry
FROM dbo.DimEmployee AS e
INNER JOIN dbo.FactSalesQuota AS sq
    ON e.EmployeeKey = sq.EmployeeKey
INNER JOIN dbo.DimSalesTerritory AS st
    ON e.SalesTerritoryKey = st.SalesTerritoryKey
WHERE sq.CalendarYear = 2003
GROUP BY e.LastName,e.SalesTerritoryKey,st.SalesTerritoryCountry
ORDER BY st.SalesTerritoryCountry, Quartile;

Here is the result set.

LastName Quartile SalesYTD SalesTerritoryCountry

----------------- -------- -------------- ------------------

Tsoflias 1 867,000.00 Australia

Saraiva 1 2,293,000.00 Canada

Varkey Chudukatil 1 2,940,000.00 France

Valdez 1 1,294,000.00 Germany

Alberts 1 651,000.00 NA

Jiang 1 544,000.00 NA

Pak 1 5,142,000.00 United Kingdom

Mensa-Annan 1 1,481,000.00 United States

Campbell 1 1,438,000.00 United States

Reiter 1 2,768,000.00 United States

Blythe 1 4,716,000.00 United States

Carson 1 4,350,000.00 United States

Mitchell 1 4,682,000.00 United States

Vargas 2 1,617,000.00 Canada

Abbas 2 172,000.00 NA

Ito 2 2,644,000.00 United States

Ansman-Wolfe 2 1,183,000.00 United States

See Also

Reference

RANK (SQL Server PDW)
DENSE_RANK (SQL Server PDW)
ROW_NUMBER (SQL Server PDW)


NULLIF (SQL Server PDW)

[image]

NULLIF (SQL Server PDW)
See Also Send Feedback

Returns a null value if the two specified expressions are equal in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

NULLIF ( expression , expression )

Arguments

expression

Is any valid scalar expression.

Return Types

Returns the same type as the first expression.

Return Values

NULLIF returns the first expression if the two expressions are not equal. If the expressions are equal, NULLIF returns a null value of the type of the first expression.

Examples

The following example creates a budgets table, loads data, and uses NULLIF to return a null if neither current_year nor previous_year contains data.

Copy image Copy Code
CREATE TABLE budgets (
   dept           tinyint,
   current_year   decimal(10,2),
   previous_year  decimal(10,2)
);

INSERT INTO budgets VALUES(1, 100000, 150000);
INSERT INTO budgets VALUES(2, NULL, 300000);
INSERT INTO budgets VALUES(3, 0, 100000);
INSERT INTO budgets VALUES(4, NULL, 150000);
INSERT INTO budgets VALUES(5, 300000, 300000);

SELECT dept, NULLIF(current_year,
   previous_year) AS LastBudget
FROM budgets;

Here is the result set.

dept LastBudget

---- -----------

1 100000.00

2 null

3 0.00

4 null

5 null

See Also

Reference

ISNULL (SQL Server PDW)
Expressions (SQL Server PDW)
NULL and UNKNOWN (SQL Server PDW)


OBJECT_ID (SQL Server PDW)

[image]

OBJECT_ID (SQL Server PDW)
See Also Send Feedback

Returns the database object identification number of a schema-scoped object.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

OBJECT_ID ( '[ database_name . [ schema_name ] . | schema_name . ]
  object_name' [ ,'object_type' ] )

Arguments

'object_name'

The object to be used. object_name is either varchar or nvarchar. If object_name is varchar, it is implicitly converted to nvarchar. Specifying the database and schema names is optional.

'object_type'

The schema-scoped object type. object_type is either varchar or nvarchar. If object_type is varchar, it is implicitly converted to nvarchar.

Return Types

int

Exceptions

Returns NULL on error.

A user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as OBJECT_ID may return NULL if the user does not have any permission on the object.

Remarks

When the parameter to a system function is optional, the current database, host computer, server user, or database user is assumed. Built-in functions must always be followed by parentheses.

System functions can be used in most places where expressions are allowed, including a SELECT list or a WHERE clause. In situations where a system function is not supported, the statement will return an error.

Examples

The following example returns the object ID for the FactFinance table in the AdventureWorksPDW2012 database.

Copy image Copy Code
SELECT OBJECT_ID(AdventureWorksPDW2012.dbo.FactFinance') AS 'Object ID';

See Also

Reference

OBJECT_NAME (SQL Server PDW)


OBJECT_NAME (SQL Server PDW)

[image]

OBJECT_NAME (SQL Server PDW)
See Also Send Feedback

Returns the database object name for schema-scoped objects.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

OBJECT_NAME ( object_id [, database_id ] )

Arguments

object_id

The ID of the object to be used. object_id is int and is assumed to be a schema-scoped object in the specified database, or in the current database context.

database_id

The ID of the database where the object is to be looked up. database_id is int.

Return Types

sysname

Exceptions

Returns NULL on error or if a caller does not have permission to view the object.

A user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as OBJECT_NAME may return NULL if the user does not have any permission on the object.

Permissions

Requires ANY permission on the object.

Remarks

System functions can be used in most places where expressions are allowed, including a SELECT list or a WHERE clause. In situations where a system function is not supported, the statement will return an error.

By default, the SQL Server PDW assumes that object_id is in the context of the current database. A query that references an object_id in another database returns NULL or incorrect results.

You can resolve object names in the context of another database by specifying a database ID. The following example specifies the database ID for the master database in the OBJECT_SCHEMA_NAME function and returns the correct results.

Examples

A. Using OBJECT_NAME in a WHERE clause

The following example returns columns from the sys.objects catalog view for the object specified by OBJECT_NAME in the WHERE clause of the SELECT statement. (Your object number (274100017 in the example below) will be different. To test this example, look up a valid object number by executing SELECT name, object_id FROM sys.objects; in your database.)

Copy image Copy Code
SELECT name, object_id, type_desc
FROM sys.objects
WHERE name = OBJECT_NAME(274100017);

See Also

Reference

OBJECT_ID (SQL Server PDW)


OBJECTPROPERTY (SQL Server PDW)

[image]

OBJECTPROPERTY (SQL Server PDW)
Send Feedback

Returns information about schema-scoped objects in the current database. For a list of schema-scoped objects, see sys.objects (SQL Server PDW). For more information, see the OBJECTPROPERTY (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

OBJECTPROPERTY ( id , property )

Arguments

id

An expression that represents the ID of the object in the current database. id is int and is assumed to be a schema-scoped object in the current database context.

property

An expression that represents the information to be returned for the object specified by id. property can be one of the following values.

Note Note

Unless noted otherwise, NULL is returned when property is not a valid property name, id is not a valid object ID, id is an unsupported object type for the specified property, or the caller does not have permission to view the object's metadata.

Property name

Object type

Description and values returned

CnstIsClustKey

Constraint

PRIMARY KEY constraint with a clustered index.

1 = True

0 = False

CnstIsColumn

Constraint

CHECK, DEFAULT, or FOREIGN KEY constraint on a single column.

1 = True

0 = False

CnstIsDeleteCascade

Constraint

FOREIGN KEY constraint with the ON DELETE CASCADE option.

1 = True

0 = False

CnstIsDisabled

Constraint

Disabled constraint.

1 = True

0 = False

CnstIsNonclustKey

Constraint

PRIMARY KEY or UNIQUE constraint with a nonclustered index.

1 = True

0 = False

CnstIsNotRepl

Constraint

Constraint is defined by using the NOT FOR REPLICATION keywords.

1 = True

0 = False

CnstIsNotTrusted

Constraint

Constraint was enabled without checking existing rows; therefore, the constraint may not hold for all rows.

1 = True

0 = False

CnstIsUpdateCascade

Constraint

FOREIGN KEY constraint with the ON UPDATE CASCADE option.

1 = True

0 = False

ExecIsAfterTrigger

Trigger

AFTER trigger.

1 = True

0 = False

ExecIsAnsiNullsOn

Transact-SQL function, Transact-SQL procedure, Transact-SQL trigger, view

Setting of ANSI_NULLS at creation time.

1 = True

0 = False

ExecIsDeleteTrigger

Trigger

DELETE trigger.

1 = True

0 = False

ExecIsFirstDeleteTrigger

Trigger

First trigger fired when a DELETE is executed against the table.

1 = True

0 = False

ExecIsFirstInsertTrigger

Trigger

First trigger fired when an INSERT is executed against the table.

1 = True

0 = False

ExecIsFirstUpdateTrigger

Trigger

First trigger fired when an UPDATE is executed against the table.

1 = True

0 = False

ExecIsInsertTrigger

Trigger

INSERT trigger.

1 = True

0 = False

ExecIsInsteadOfTrigger

Trigger

INSTEAD OF trigger.

1 = True

0 = False

ExecIsLastDeleteTrigger

Trigger

Last trigger fired when a DELETE is executed against the table.

1 = True

0 = False

ExecIsLastInsertTrigger

Trigger

Last trigger fired when an INSERT is executed against the table.

1 = True

0 = False

ExecIsLastUpdateTrigger

Trigger

Last trigger fired when an UPDATE is executed against the table.

1 = True

0 = False

ExecIsQuotedIdentOn

Transact-SQL function, Transact-SQL procedure, Transact-SQL trigger, view

Setting of QUOTED_IDENTIFIER at creation time.

1 = True

0 = False

ExecIsStartup

Procedure

Startup procedure.

1 = True

0 = False

ExecIsTriggerDisabled

Trigger

Disabled trigger.

1 = True

0 = False

ExecIsTriggerNotForRepl

Trigger

Trigger defined as NOT FOR REPLICATION.

1 = True

0 = False

ExecIsUpdateTrigger

Trigger

UPDATE trigger.

1 = True

0 = False

HasAfterTrigger

Table, view

Table or view has an AFTER trigger.

1 = True

0 = False

HasDeleteTrigger

Table, view

Table or view has a DELETE trigger.

1 = True

0 = False

HasInsertTrigger

Table, view

Table or view has an INSERT trigger.

1 = True

0 = False

HasInsteadOfTrigger

Table, view

Table or view has an INSTEAD OF trigger.

1 = True

0 = False

HasUpdateTrigger

Table, view

Table or view has an UPDATE trigger.

1 = True

0 = False

IsAnsiNullsOn

Transact-SQL function, Transact-SQL procedure, table, Transact-SQL trigger, view

Specifies that the ANSI NULLS option setting for the table is ON. This means all comparisons against a null value evaluate to UNKNOWN. This setting applies to all expressions in the table definition, including computed columns and constraints, for as long as the table exists.

1 = True

0 = False

IsCheckCnst

Any schema-scoped object

CHECK constraint.

1 = True

0 = False

IsConstraint

Any schema-scoped object

Is a single column CHECK, DEFAULT, or FOREIGN KEY constraint on a column or table.

1 = True

0 = False

IsDefault

Any schema-scoped object

Bound default.

1 = True

0 = False

IsDefaultCnst

Any schema-scoped object

DEFAULT constraint.

1 = True

0 = False

IsDeterministic

Function, view

The determinism property of the function or view.

1 = Deterministic

0 = Not Deterministic

IsEncrypted

Transact-SQL function, Transact-SQL procedure, table, Transact-SQL trigger, view

Indicates that the original text of the module statement was converted to an obfuscated format. The output of the obfuscation is not directly visible in any of the catalog views in SQL Server 2005. Users without access to system tables or database files cannot retrieve the obfuscated text. However, the text is available to users that can either access system tables over the DAC port or directly access database files. Also, users that can attach a debugger to the server process can retrieve the original procedure from memory at run time.

1 = Encrypted

0 = Not encrypted

Base data type: int

IsExecuted

Any schema-scoped object

Object can be executed (view, procedure, function, or trigger).

1 = True

0 = False

IsExtendedProc

Any schema-scoped object

Extended procedure.

1 = True

0 = False

IsForeignKey

Any schema-scoped object

FOREIGN KEY constraint.

1 = True

0 = False

IsIndexed

Table, view

Table or view that has an index.

1 = True

0 = False

IsIndexable

Table, view

Table or view on which an index can be created.

1 = True

0 = False

IsInlineFunction

Function

Inline function.

1 = Inline function

0 = Not inline function

IsMSShipped

Any schema-scoped object

Object created during installation of SQL Server.

1 = True

0 = False

IsPrimaryKey

Any schema-scoped object

PRIMARY KEY constraint.

1 = True

0 = False

NULL = Not a function, or object ID is not valid.

IsProcedure

Any schema-scoped object

Procedure.

1 = True

0 = False

IsQuotedIdentOn

Transact-SQL function, Transact-SQL procedure, table, Transact-SQL trigger, view, CHECK constraint, DEFAULT definition

Specifies that the quoted identifier setting for the object is ON. This means double quotation marks delimit identifiers in all expressions involved in the object definition.

1 = ON

0 = OFF

IsQueue

Any schema-scoped object

Service Broker Queue

1 = True

0 = False

IsReplProc

Any schema-scoped object

Replication procedure.

1 = True

0 = False

IsRule

Any schema-scoped object

Bound rule.

1 = True

0 = False

IsScalarFunction

Function

Scalar-valued function.

1 = Scalar-valued function

0 = Not scalar-valued function

IsSchemaBound

Function, view

A schema bound function or view created by using SCHEMABINDING.

1 = Schema-bound

0 = Not schema-bound.

IsSystemTable

Table

System table.

1 = True

0 = False

IsTable

Table

Table.

1 = True

0 = False

IsTableFunction

Function

Table-valued function.

1 = Table-valued function

0 = Not table-valued function

IsTrigger

Any schema-scoped object

Trigger.

1 = True

0 = False

IsUniqueCnst

Any schema-scoped object

UNIQUE constraint.

1 = True

0 = False

IsUserTable

Table

User-defined table.

1 = True

0 = False

IsView

View

View.

1 = True

0 = False

OwnerId

Any schema-scoped object

Owner of the object.

Note Note

The schema owner is not necessarily the object owner. For example, child objects (those where parent_object_id is nonnull) will always return the same owner ID as the parent.

Nonnull = The database user ID of the object owner.

TableDeleteTrigger

Table

Table has a DELETE trigger.

>1 = ID of first trigger with the specified type.

TableDeleteTriggerCount

Table

Table has the specified number of DELETE triggers.

>0 = The number of DELETE triggers.

TableFullTextMergeStatus

Table

Whether a table that has a full-text index that is currently in merging.

0 = Table does not have a full-text index, or the full-text index is not in merging.

1 = The full-text index is in merging.

TableFullTextBackgroundUpdateIndexOn

Table

Table has full-text background update index (autochange tracking) enabled.

1 = TRUE

0 = FALSE

TableFulltextCatalogId

Table

ID of the full-text catalog in which the full-text index data for the table resides.

Nonzero = Full-text catalog ID, associated with the unique index that identifies the rows in a full-text indexed table.

0 = Table does not have a full-text index.

TableFulltextChangeTrackingOn

Table

Table has full-text change-tracking enabled.

1 = TRUE

0 = FALSE

TableFulltextDocsProcessed

Table

Number of rows processed since the start of full-text indexing. In a table that is being indexed for full-text search, all the columns of one row are considered as part of one document to be indexed.

0 = No active crawl or full-text indexing is completed.

> 0 = One of the following:

  • The number of documents processed by insert or update operations since the start of Full, Incremental, or Manual change tracking population.

  • The number of rows processed by insert or update operations since change tracking with background update index population was enabled, the full-text index schema changed, the full-text catalog rebuilt, or the instance of SQL Server restarted, and so on.

NULL = Table does not have a full-text index.

Note Note

This property does not monitor or count deleted rows.

TableFulltextFailCount

Table

Number of rows Full-Text Search did not index.

0 = The population has completed.

> 0 = One of the following:

  • The number of documents that were not indexed since the start of Full, Incremental, and Manual Update change tracking population.

  • For change tracking with background update index, the number of rows that were not indexed since the start of the population, or the restart of the population. This could be caused by a schema change, rebuild of the catalog, server restart, and so on.

NULL = Table does not have a full-text index.

TableFulltextItemCount

Table

Number of rows that were successfully full-text indexed.

TableFulltextKeyColumn

Table

ID of the column associated with the single-column unique index that is participating in the full-text index definition.

0 = Table does not have a full-text index.

TableFulltextPendingChanges

Table

Number of pending change tracking entries to process.

0 = change tracking is not enabled.

NULL = Table does not have a full-text index.

TableFulltextPopulateStatus

Table

0 = Idle.

1 = Full population is in progress.

2 = Incremental population is in progress.

3 = Propagation of tracked changes is in progress.

4 = Background update index is in progress, such as autochange tracking.

5 = Full-text indexing is throttled or paused.

TableHasActiveFulltextIndex

Table

Table has an active full-text index.

1 = True

0 = False

TableHasCheckCnst

Table

Table has a CHECK constraint.

1 = True

0 = False

TableHasClustIndex

Table

Table has a clustered index.

1 = True

0 = False

TableHasDefaultCnst

Table

Table has a DEFAULT constraint.

1 = True

0 = False

TableHasDeleteTrigger

Table

Table has a DELETE trigger.

1 = True

0 = False

TableHasForeignKey

Table

Table has a FOREIGN KEY constraint.

1 = True

0 = False

TableHasForeignRef

Table

Table is referenced by a FOREIGN KEY constraint.

1 = True

0 = False

TableHasIdentity

Table

Table has an identity column.

1 = True

0 = False

TableHasIndex

Table

Table has an index of any type.

1 = True

0 = False

TableHasInsertTrigger

Table

Object has an INSERT trigger.

1 = True

0 = False

TableHasNonclustIndex

Table

Table has a nonclustered index.

1 = True

0 = False

TableHasPrimaryKey

Table

Table has a primary key.

1 = True

0 = False

TableHasRowGuidCol

Table

Table has a ROWGUIDCOL for a uniqueidentifier column.

1 = True

0 = False

TableHasTextImage

Table

Table has a text, ntext, or image column.

1 = True

0 = False

TableHasTimestamp

Table

Table has a timestamp column.

1 = True

0 = False

TableHasUniqueCnst

Table

Table has a UNIQUE constraint.

1 = True

0 = False

TableHasUpdateTrigger

Table

Object has an UPDATE trigger.

1 = True

0 = False

TableHasVarDecimalStorageFormat

Table

Table is enabled for vardecimal storage format.

1 = True

0 = False

TableInsertTrigger

Table

Table has an INSERT trigger.

>1 = ID of first trigger with the specified type.

TableInsertTriggerCount

Table

Table has the specified number of INSERT triggers.

>0 = The number of INSERT triggers.

TableIsFake

Table

Table is not real. It is materialized internally on demand by the SQL Server Database Engine.

1 = True

0 = False

TableIsLockedOnBulkLoad

Table

Table is locked due to a bcp or BULK INSERT job.

1 = True

0 = False

Not available in SQL Server PDW.

TableIsPinned

Table

Table is pinned to be held in the data cache.

0 = False

This feature is not supported in SQL Server 2005 and later.

TableTextInRowLimit

Table

Maximum bytes allowed for text in row.

0 if text in row option is not set.

TableUpdateTrigger

Table

Table has an UPDATE trigger.

> 1 = ID of first trigger with the specified type.

TableUpdateTriggerCount

Table

The table has the specified number of UPDATE triggers.

> 0 = The number of UPDATE triggers.

TableHasColumnSet

Table

Table has a column set.

0 = False

1 = True

For more information, see Use Column Sets.

Return Types

int

Exceptions

Returns NULL on error or if a caller does not have permission to view the object.

A user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as OBJECTPROPERTY may return NULL if the user does not have any permission on the object.

Remarks

Returns an error if the property is valid in SQL Server but not valid for SQL Server PDW.

The SQL Server PDW assumes that object_id is in the current database context. A query that references an object_id in another database will return NULL or incorrect results.

OBJECTPROPERTY(view_id, 'IsIndexable') may consume significant computer resources because evaluation of IsIndexable property requires the parsing of view definition, normalization, and partial optimization. Although the IsIndexable property identifies tables or views that can be indexed, the actual creation of the index still might fail if certain index key requirements are not met.

OBJECTPROPERTY(table_id, 'TableHasActiveFulltextIndex') will return a value of 1 (true) when at least one column of a table is added for indexing. Full-text indexing becomes active for population as soon as the first column is added for indexing.

Properties that are valid but not applicable to SQL Server PDW such as properties related to triggers, always return 0.

Examples

The following example tests whether dbo.DimReseller is a table in the AdventureWorksPDW2012 database.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
GO
IF OBJECTPROPERTY (OBJECT_ID(N'dbo.DimReseller'),'ISTABLE') = 1
   SELECT 'DimReseller is a table.'
ELSE
   SELECT 'DimReseller is not a table.';
GO

The following example returns all the tables in the dbo schema.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
GO
SELECT name, object_id, type_desc
FROM sys.objects
WHERE OBJECTPROPERTY(object_id, N'SchemaId') = SCHEMA_ID(N'dbo')
ORDER BY type_desc, name;
GO


OBJECTPROPERTYEX (SQL Server PDW)

[image]

OBJECTPROPERTYEX (SQL Server PDW)
Send Feedback

Returns information about schema-scoped objects in the current database. For a list of these objects, see sys.objects (SQL Server PDW). For more information, see the OBJECTPROPERTYEX (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

OBJECTPROPERTYEX ( id , property )

Arguments

id

The ID of the object in the current database. id is int and is assumed to be a schema-scoped object in the current database context.

property

The information to be returned for the object specified by id. The return type is sql_variant. The following table shows the base data type for each property value.

Note Note

Unless noted otherwise, NULL is returned when property is not a valid property name, id is not a valid object ID, id is an unsupported object type for the specified property, or the caller does not have permission to view the object's metadata.

Property name

Object type

Description and values returned

BaseType

Any schema-scoped object

Identifies the base type of the object. When the specified object is a SYNONYM, the base type of the underlying object is returned.

Nonnull = Object type

Base data type: char(2)

CnstIsClustKey

Constraint

PRIMARY KEY constraint with a clustered index.

1 = True

0 = False

Base data type: int

CnstIsColumn

Constraint

CHECK, DEFAULT, or FOREIGN KEY constraint on a single column.

1 = True

0 = False

Base data type: int

CnstIsDeleteCascade

Constraint

FOREIGN KEY constraint with the ON DELETE CASCADE option.

1 = True

0 = False

Base data type: int

CnstIsDisabled

Constraint

Disabled constraint.

1 = True

0 = False

Base data type: int

CnstIsNonclustKey

Constraint

PRIMARY KEY constraint with a nonclustered index.

1 = True

0 = False

Base data type: int

CnstIsNotRepl

Constraint

Constraint is defined by using the NOT FOR REPLICATION keywords.

1 = True

0 = False

Base data type: int

CnstIsNotTrusted

Constraint

Constraint was enabled without checking existing rows. Therefore, the constraint may not hold for all rows.

1 = True

0 = False

Base data type: int

CnstIsUpdateCascade

Constraint

FOREIGN KEY constraint with the ON UPDATE CASCADE option.

1 = True

0 = False

Base data type: int

ExecIsAfterTrigger

Trigger

AFTER trigger.

1 = True

0 = False

Base data type: int

ExecIsAnsiNullsOn

Transact-SQL function, Transact-SQL procedure, Transact-SQL trigger, view

The setting of ANSI_NULLS at creation time.

1 = True

0 = False

Base data type: int

ExecIsDeleteTrigger

Trigger

DELETE trigger.

1 = True

0 = False

Base data type: int

ExecIsFirstDeleteTrigger

Trigger

The first trigger fired when a DELETE is executed against the table.

1 = True

0 = False

Base data type: int

ExecIsFirstInsertTrigger

Trigger

The first trigger fired when an INSERT is executed against the table.

1 = True

0 = False

Base data type: int

ExecIsFirstUpdateTrigger

Trigger

The first trigger fired when an UPDATE is executed against the table.

1 = True

0 = False

Base data type: int

ExecIsInsertTrigger

Trigger

INSERT trigger.

1 = True

0 = False

Base data type: int

ExecIsInsteadOfTrigger

Trigger

INSTEAD OF trigger.

1 = True

0 = False

Base data type: int

ExecIsLastDeleteTrigger

Trigger

Last trigger fired when a DELETE is executed against the table.

1 = True

0 = False

Base data type: int

ExecIsLastInsertTrigger

Trigger

Last trigger fired when an INSERT is executed against the table.

1 = True

0 = False

Base data type: int

ExecIsLastUpdateTrigger

Trigger

Last trigger fired when an UPDATE is executed against the table.

1 = True

0 = False

Base data type: int

ExecIsQuotedIdentOn

Transact-SQL function, Transact-SQL procedure, Transact-SQL trigger, view

Setting of QUOTED_IDENTIFIER at creation time.

1 = True

0 = False

Base data type: int

ExecIsStartup

Procedure

Startup procedure.

1 = True

0 = False

Base data type: int

ExecIsTriggerDisabled

Trigger

Disabled trigger.

1 = True

0 = False

Base data type: int

ExecIsTriggerNotForRepl

Trigger

Trigger defined as NOT FOR REPLICATION.

1 = True

0 = False

Base data type: int

ExecIsUpdateTrigger

Trigger

UPDATE trigger.

1 = True

0 = False

Base data type: int

HasAfterTrigger

Table, view

Table or view has an AFTER trigger.

1 = True

0 = False

Base data type: int

HasDeleteTrigger

Table, view

Table or view has a DELETE trigger.

1 = True

0 = False

Base data type: int

HasInsertTrigger

Table, view

Table or view has an INSERT trigger.

1 = True

0 = False

Base data type: int

HasInsteadOfTrigger

Table, view

Table or view has an INSTEAD OF trigger.

1 = True

0 = False

Base data type: int

HasUpdateTrigger

Table, view

Table or view has an UPDATE trigger.

1 = True

0 = False

Base data type: int

IsAnsiNullsOn

Transact-SQL function, Transact-SQL procedure, table, Transact-SQL trigger, view

Specifies that the ANSI NULLS option setting for the table is ON, meaning all comparisons against a null value evaluate to UNKNOWN. This setting applies to all expressions in the table definition, including computed columns and constraints, for as long as the table exists.

1 = True

0 = False

Base data type: int

IsCheckCnst

Any schema-scoped object

CHECK constraint.

1 = True

0 = False

Base data type: int

IsConstraint

Any schema-scoped object

Constraint.

1 = True

0 = False

Base data type: int

IsDefault

Any schema-scoped object

Bound default.

1 = True

0 = False

Base data type: int

IsDefaultCnst

Any schema-scoped object

DEFAULT constraint.

1 = True

0 = False

Base data type: int

IsDeterministic

Scalar and table-valued functions, view

The determinism property of the function or view.

1 = Deterministic

0 = Not Deterministic

Base data type: int

IsEncrypted

Transact-SQL function, Transact-SQL procedure, table, Transact-SQL trigger, view

Indicates that the original text of the module statement was converted to an obfuscated format. The output of the obfuscation is not directly visible in any of the catalog views in SQL Server 2005. Users without access to system tables or database files cannot retrieve the obfuscated text. However, the text is available to users that can either access system tables over the DAC port or directly access database files. Also, users that can attach a debugger to the server process can retrieve the original procedure from memory at run time.

1 = Encrypted

0 = Not encrypted

Base data type: int

IsExecuted

Any schema-scoped object

Specifies the object can be executed (view, procedure, function, or trigger).

1 = True

0 = False

Base data type: int

IsExtendedProc

Any schema-scoped object

Extended procedure.

1 = True

0 = False

Base data type: int

IsForeignKey

Any schema-scoped object

FOREIGN KEY constraint.

1 = True

0 = False

Base data type: int

IsIndexed

Table, view

A table or view with an index.

1 = True

0 = False

Base data type: int

IsIndexable

Table, view

A table or view on which an index may be created.

1 = True

0 = False

Base data type: int

IsInlineFunction

Function

Inline function.

1 = Inline function

0 = Not inline function

Base data type: int

IsMSShipped

Any schema-scoped object

An object created during installation of SQL Server.

1 = True

0 = False

Base data type: int

IsPrecise

Computed column, function, user-defined type, view

Indicates whether the object contains an imprecise computation, such as floating point operations.

1 = Precise

0 = Imprecise

Base data type: int

IsPrimaryKey

Any schema-scoped object

PRIMARY KEY constraint.

1 = True

0 = False

Base data type: int

IsProcedure

Any schema-scoped object

Procedure.

1 = True

0 = False

Base data type: int

IsQuotedIdentOn

CHECK constraint, DEFAULT definition, Transact-SQL function, Transact-SQL procedure, table, Transact-SQL trigger, view

Specifies that the quoted identifier setting for the object is ON, meaning double quotation marks delimit identifiers in all expressions involved in the object definition.

1 = True

0 = False

Base data type: int

IsQueue

Any schema-scoped object

Service Broker Queue

1 = True

0 = False

Base data type: int

IsReplProc

Any schema-scoped object

Replication procedure.

1 = True

0 = False

Base data type: int

IsRule

Any schema-scoped object

Bound rule.

1 = True

0 = False

Base data type: int

IsScalarFunction

Function

Scalar-valued function.

1 = Scalar-valued function

0 = Not scalar-valued function

Base data type: int

IsSchemaBound

Function, view

A schema bound function or view created by using SCHEMABINDING.

1 = Schema-bound

0 = Not schema-bound

Base data type: int

IsSystemTable

Table

System table.

1 = True

0 = False

Base data type: int

IsSystemVerified

Computed column, function, user-defined type, view

The precision and determinism properties of the object can be verified by SQL Server.

1 = True

0 = False

Base data type: int

IsTable

Table

Table.

1 = True

0 = False

Base data type: int

IsTableFunction

Function

Table-valued function.

1 = Table-valued function

0 = Not table-valued function

Base data type: int

IsTrigger

Any schema-scoped object

Trigger.

1 = True

0 = False

Base data type: int

IsUniqueCnst

Any schema-scoped object

UNIQUE constraint.

1 = True

0 = False

Base data type: int

IsUserTable

Table

User-defined table.

1 = True

0 = False

Base data type: int

IsView

View

View.

1 = True

0 = False

Base data type: int

OwnerId

Any schema-scoped object

Owner of the object.

Note Note

The schema owner is not necessarily the object owner. For example, child objects (those where parent_object_id is nonnull) will always return the same owner ID as the parent.

Nonnull = Database user ID of the object owner.

NULL = Unsupported object type, or object ID is not valid.

Base data type: int

SchemaId

Any schema-scoped object

The ID of the schema associated with the object.

Nonnull = Schema ID of the object.

Base data type: int

SystemDataAccess

Function, view

Object accesses system data, system catalogs or virtual system tables, in the local instance of SQL Server.

0 = None

1 = Read

Base data type: int

TableDeleteTrigger

Table

Table has a DELETE trigger.

>1 = ID of first trigger with the specified type.

Base data type: int

TableDeleteTriggerCount

Table

The table has the specified number of DELETE triggers.

Nonnull = Number of DELETE triggers

Base data type: int

TableFullTextMergeStatus

Table

Whether a table that has a full-text index that is currently in merging.

0 = Table does not have a full-text index, or the full-text index is not in merging.

1 = The full-text index is in merging.

TableFullTextBackgroundUpdateIndexOn

Table

The table has full-text background update index (autochange tracking) enabled.

1 = TRUE

0 = FALSE

Base data type: int

TableFulltextCatalogId

Table

ID of the full-text catalog in which the full-text index data for the table resides.

Nonzero = Full-text catalog ID, associated with the unique index that identifies the rows in a full-text indexed table.

0 = Table does not have a full-text index.

Base data type: int

TableFullTextChangeTrackingOn

Table

Table has full-text change-tracking enabled.

1 = TRUE

0 = FALSE

Base data type: int

TableFulltextDocsProcessed

Table

Number of rows processed since the start of full-text indexing. In a table that is being indexed for full-text search, all the columns of one row are considered as part of one document to be indexed.

0 = No active crawl or full-text indexing is completed.

> 0 = One of the following:

  • The number of documents processed by insert or update operations since the start of full, incremental, or manual change tracking population.

  • The number of rows processed by insert or update operations since change tracking with background update index population was enabled, the full-text index schema changed, the full-text catalog rebuilt, or the instance of SQL Server restarted, and so on.

NULL = Table does not have a full-text index.

Base data type: int

Note This property does not monitor or count deleted rows.

TableFulltextFailCount

Table

The number of rows that full-text search did not index.

0 = The population has completed.

>0 = One of the following:

  • The number of documents that were not indexed since the start of Full, Incremental, and Manual Update change tracking population.

  • For change tracking with background update index, the number of rows that were not indexed since the start of the population, or the restart of the population. This could be caused by a schema change, rebuild of the catalog, server restart, and so on

NULL = Table does not have a Full-Text index.

Base data type: int

TableFulltextItemCount

Table

Nonnull = Number of rows that were full-text indexed successfully.

NULL = Table does not have a full-text index.

Base data type: int

TableFulltextKeyColumn

Table

ID of the column associated with the single-column unique index that is part of the definition of a full-text index and semantic index.

0 = Table does not have a full-text index.

Base data type: int

TableFulltextPendingChanges

Table

Number of pending change tracking entries to process.

0 = change tracking is not enabled.

NULL = Table does not have a full-text index.

Base data type: int

TableFulltextPopulateStatus

Table

0 = Idle.

1 = Full population is in progress.

2 = Incremental population is in progress.

3 = Propagation of tracked changes is in progress.

4 = Background update index is in progress, such as autochange tracking.

5 = Full-text indexing is throttled or paused.

Base data type: int

TableFullTextSemanticExtraction

Table

Table is enabled for semantic indexing.

1 = True

0 = False

Base data type: int

TableHasActiveFulltextIndex

Table

Table has an active full-text index.

1 = True

0 = False

Base data type: int

TableHasCheckCnst

Table

Table has a CHECK constraint.

1 = True

0 = False

Base data type: int

TableHasClustIndex

Table

Table has a clustered index.

1 = True

0 = False

Base data type: int

TableHasDefaultCnst

Table

Table has a DEFAULT constraint.

1 = True

0 = False

Base data type: int

TableHasDeleteTrigger

Table

Table has a DELETE trigger.

1 = True

0 = False

Base data type: int

TableHasForeignKey

Table

Table has a FOREIGN KEY constraint.

1 = True

0 = False

Base data type: int

TableHasForeignRef

Table

Table is referenced by a FOREIGN KEY constraint.

1 = True

0 = False

Base data type: int

TableHasIdentity

Table

Table has an identity column.

1 = True

0 = False

Base data type: int

TableHasIndex

Table

Table has an index of any type.

1 = True

0 = False

Base data type: int

TableHasInsertTrigger

Table

Object has an INSERT trigger.

1 = True

0 = False

Base data type: int

TableHasNonclustIndex

Table

The table has a nonclustered index.

1 = True

0 = False

Base data type: int

TableHasPrimaryKey

Table

Table has a primary key.

1 = True

0 = False

Base data type: int

TableHasRowGuidCol

Table

Table has a ROWGUIDCOL for a uniqueidentifier column.

1 = True

0 = False

Base data type: int

TableHasTextImage

Table

Table has a text, ntext, or image column.

1 = True

0 = False

Base data type: int

TableHasTimestamp

Table

Table has a timestamp column.

1 = True

0 = False

Base data type: int

TableHasUniqueCnst

Table

Table has a UNIQUE constraint.

1 = True

0 = False

Base data type: int

TableHasUpdateTrigger

Table

The object has an UPDATE trigger.

1 = True

0 = False

Base data type: int

TableHasVarDecimalStorageFormat

Table

Table is enabled for vardecimal storage format.

1 = True

0 = False

TableInsertTrigger

Table

Table has an INSERT trigger.

>1 = ID of first trigger with the specified type.

Base data type: int

TableInsertTriggerCount

Table

The table has the specified number of INSERT triggers.

>0 = The number of INSERT triggers.

Base data type: int

TableIsFake

Table

Table is not real. It is materialized internally on demand by the Database Engine.

1 = True

0 = False

Base data type: int

TableIsLockedOnBulkLoad

Table

Table is locked because a bcp or BULK INSERT job.

1 = True

0 = False

Base data type: int

Not supported in SQL Server PDW.

TableIsPinned

Table

Table is pinned to be held in the data cache.

0 = False

This feature is not supported in SQL Server 2005 and later versions.

TableTextInRowLimit

Table

Table has text in row option set.

> 0 = Maximum bytes allowed for text in row.

0 = text in row option is not set.

Base data type: int

TableUpdateTrigger

Table

Table has an UPDATE trigger.

> 1 = ID of first trigger with the specified type.

Base data type: int

TableUpdateTriggerCount

Table

Table has the specified number of UPDATE triggers.

> 0 = The number of UPDATE triggers.

Base data type: int

UserDataAccess

Function, View

Indicates the object accesses user data, user tables, in the local instance of SQL Server.

1 = Read

0 = None

Base data type: int

TableHasColumnSet

Table

Table has a column set.

0 = False

1 = True

For more information, see Using Column sets.

Cardinality

Table (system or user-defined), view, or index

The number of rows in the specified object. Not supported in SQL Server PDW.

Return Types

sql_variant

Exceptions

Returns NULL on error or if a caller does not have permission to view the object.

A user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as OBJECTPROPERTYEX may return NULL if the user does not have any permission on the object.

Remarks

Returns an error if the property is valid in SQL Server but not valid for SQL Server PDW.

The SQL Server PDW assumes that object_id is in the current database context. A query that references an object_id in another database will return NULL or incorrect results.

OBJECTPROPERTYEX(view_id, 'IsIndexable') may consume significant computer resources because evaluation of IsIndexable property requires the parsing of view definition, normalization, and partial optimization

OBJECTPROPERTYEX (table_id, 'TableHasActiveFulltextIndex') will return a value of 1 (true) when at least one column of a table is added for indexing. Full-text indexing becomes active for population as soon as the first column is added for indexing.

Properties that are valid but not applicable to SQL Server PDW such as properties related to triggers, always return 0.

Examples

The following example creates returns the base type of dbo.DimReseller object.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
GO
SELECT OBJECTPROPERTYEX ( object_id(N'dbo.DimReseller'), N'BaseType')AS BaseType;

The result set shows that the base type of the underlying object, the dbo.DimReseller table, is a user table.

Copy image Copy Code
BaseType
--------
U


ODBC Scalar Functions (SQL Server PDW)

[image]

ODBC Scalar Functions (SQL Server PDW)
See Also Send Feedback

You can use ODBC Scalar Functions in Transact-SQL statements. These statements are interpreted by SQL Server. They can be used in stored procedures and user-defined functions. These include string, numeric, time, date, interval, and system functions.

Topic link icon Syntax Conventions (SQL Server PDW)

Usage

SELECT {fn <function_name> [ (<argument>,....n) ] }

Functions

The following tables list ODBC scalar functions that are not duplicated in Transact-SQL.

String Functions

Function | Description
BIT_LENGTH( string_exp ) (ODBC 3.0) | Returns the length in bits of the string expression. Does not work only for string data types. Therefore, will not implicitly convert string_exp to string but instead will return the (internal) size of whatever data type it is given.
CONCAT( string_exp1,string_exp2) (ODBC 1.0) | Returns a character string that is the result of concatenating string_exp2 to string_exp1. The resulting string is DBMS-dependent. For example, if the column represented by string_exp1 contained a NULL value, DB2 would return NULL but SQL Server would return the non-NULL string.

Time, Date, and Interval Functions

Function | Description
CURRENT_DATE( ) (ODBC 3.0) | Returns the current date.
CURDATE( ) (ODBC 3.0) | Returns the current date.
CURRENT_TIME() (ODBC 3.0) | Returns the current local time. APS does not support the time-precision argument which determines the seconds precision of the returned value. In APS, CURRENT_TIME takes 0 arguments.
CURTIME() (ODBC 3.0) | Returns the current local time.
DAYNAME( date_exp ) (ODBC 2.0) | Returns a character string that contains the data source–specific name of the day (for example, Sunday through Saturday or Sun. through Sat. for a data source that uses English, or Sonntag through Samstag for a data source that uses German) for the day part of date_exp.
DAYOFMONTH( date_exp ) (ODBC 1.0) | Returns the day of the month based on the month field in date_exp as an integer value in the range of 1–31.
DAYOFWEEK( date_exp ) (ODBC 1.0) | Returns the day of the week based on the week field in date_exp as an integer value in the range of 1–7, where 1 represents Sunday.
HOUR( time_exp ) (ODBC 1.0) | Returns the hour based on the hour field in time_exp as an integer value in the range of 0–23.
MINUTE( time_exp ) (ODBC 1.0) | Returns the minute based on the minute field in time_exp as an integer value in the range of 0–59.
SECOND( time_exp ) (ODBC 1.0) | Returns the second based on the second field in time_exp as an integer value in the range of 0–59.
MONTHNAME( date_exp ) (ODBC 2.0) | Returns a character string that contains the data source–specific name of the month (for example, January through December or Jan. through Dec. for a data source that uses English, or Januar through Dezember for a data source that uses German) for the month part of date_exp.
QUARTER( date_exp ) (ODBC 1.0) | Returns the quarter in date_exp as an integer value in the range of 1–4, where 1 represents January 1 through March 31.
WEEK( date_exp ) (ODBC 1.0) | Returns the week of the year based on the week field in date_exp as an integer value in the range of 1–53.

Examples

A. Using an ODBC function in a stored procedure

The following example uses an ODBC function in a stored procedure:

Copy image Copy Code
CREATE PROCEDURE dbo.ODBCprocedure
    (
    @string_exp nvarchar(4000)
    )
AS
SELECT {fn BIT_LENGTH( @string_exp )};

B. Using an ODBC Function in a user-defined function

The following example uses an ODBC function in a user-defined function:

Copy image Copy Code
CREATE FUNCTION dbo.ODBCudf
    (
    @string_exp nvarchar(4000)
    )
RETURNS int
AS
BEGIN
DECLARE @len int
SET @len = (SELECT {fn BIT_LENGTH( @string_exp )})
RETURN(@len)
END ;

SELECT dbo.ODBCudf('Returns the length in bits.');
--Returns 432

C. Using an ODBC functions in SELECT statements

The following SELECT statements use ODBC functions:

Copy image Copy Code
DECLARE @string_exp nvarchar(4000) = 'Returns the length.';
SELECT {fn BIT_LENGTH( @string_exp )};
-- Returns 304

SELECT {fn CONCAT( 'CONCAT ','returns a character string')};
-- Returns CONCAT returns a character string
SELECT {fn CURRENT_DATE( )};
-- Returns todays date
SELECT {fn CURRENT_TIME(6)};
-- Returns the time

DECLARE @date_exp nvarchar(30) = '2007-04-21 01:01:01.1234567';
SELECT {fn DAYNAME( @date_exp )};
-- Returns Saturday
SELECT {fn DAYOFMONTH( @date_exp )};
-- Returns 21
SELECT {fn DAYOFWEEK( @date_exp )};
-- Returns 7
SELECT {fn HOUR( @date_exp)};
-- Returns 1
SELECT {fn MINUTE( @date_exp )};
-- Returns 1
SELECT {fn SECOND( @date_exp )};
-- Returns 1
SELECT {fn MONTHNAME( @date_exp )};
-- Returns April
SELECT {fn QUARTER( @date_exp )};
-- Returns 2
SELECT {fn WEEK( @date_exp )};
-- Returns 16

See Also

Reference

Functions (SQL Server PDW)


OVER Clause (SQL Server PDW)

[image]

OVER Clause (SQL Server PDW)
See Also Send Feedback

Determines the partitioning and ordering of the rowset before the analytic function is applied.

For more information, see Functions (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

        OVER ( [ PARTITION BY value_expression ] [ order_by_clause ] )

Arguments

PARTITION BY

Divides the result set into partitions. The analytic function is applied to each partition separately and computation restarts for each partition.

value_expression

Specifies the column by which the rowset produced by the corresponding FROM clause is partitioned. value_expression can only refer to columns made available by the FROM clause. value_expression cannot refer to expressions or aliases in the select list. value_expression can be a column expression, scalar subquery, or scalar function.

order_by_clause

Specifies the order to apply the analytic function. For more information, see ORDER BY.

Important note Important

When used in the context of an analytic function, order_by_clause can only refer to columns made available by the FROM clause. An integer cannot be specified to represent the position of the name or alias of a column in the select list.

Remarks

More than one analytic function can be used in a single query with a single FROM clause. However, the OVER clause for each function can differ in partitioning and also ordering.

Examples

A. Using the OVER clause with the ROW_NUMBER function

The following example returns the ROW_NUMBER for sales representatives based on their assigned sales quota.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT ROW_NUMBER() OVER(ORDER BY SUM(SalesAmountQuota) DESC) AS RowNumber,
    FirstName, LastName,
CONVERT(varchar(13), SUM(SalesAmountQuota),1) AS SalesQuota
FROM dbo.DimEmployee AS e
INNER JOIN dbo.FactSalesQuota AS sq
    ON e.EmployeeKey = sq.EmployeeKey
WHERE e.SalesPersonFlag = 1
GROUP BY LastName, FirstName;

Here is a partial result set.

RowNumber FirstName LastName SalesQuota

--------- --------- ------------------ -------------

1 Jillian Carson 12,198,000.00

2 Linda Mitchell 11,786,000.00

3 Michael Blythe 11,162,000.00

4 Jae Pak 10,514,000.00

B. Using the OVER clause with aggregate functions

The following examples show using the OVER clause with aggregate functions. In this example, using the OVER clause is more efficient than using subqueries.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT SalesOrderNumber AS OrderNumber, ProductKey,
       OrderQuantity AS Qty,
       SUM(OrderQuantity) OVER(PARTITION BY SalesOrderNumber) AS Total,
       AVG(OrderQuantity) OVER(PARTITION BY SalesOrderNumber) AS Avg,
       COUNT(OrderQuantity) OVER(PARTITION BY SalesOrderNumber) AS Count,
       MIN(OrderQuantity) OVER(PARTITION BY SalesOrderNumber) AS Min,
       MAX(OrderQuantity) OVER(PARTITION BY SalesOrderNumber) AS Max
FROM dbo.FactResellerSales
WHERE SalesOrderNumber IN(N'SO43659',N'SO43664') AND
      ProductKey LIKE '2%'
ORDER BY SalesOrderNumber,ProductKey;

Here is the result set.

OrderNumber Product Qty Total Avg Count Min Max

----------- ------- --- ----- --- ----- --- ---

SO43659 218 6 16 3 5 1 6

SO43659 220 4 16 3 5 1 6

SO43659 223 2 16 3 5 1 6

SO43659 229 3 16 3 5 1 6

SO43659 235 1 16 3 5 1 6

SO43664 229 1 2 1 2 1 1

SO43664 235 1 2 1 2 1 1

The following example shows using the OVER clause with an aggregate function in a calculated value. Notice that the aggregates are calculated by SalesOrderNumber and the percentage of the total sales order is calculated for each line of each SalesOrderNumber.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT SalesOrderNumber AS OrderNumber, ProductKey AS Product,
       OrderQuantity AS Qty,
       SUM(OrderQuantity) OVER(PARTITION BY SalesOrderNumber) AS Total,
       CAST(1. * OrderQuantity / SUM(OrderQuantity)
        OVER(PARTITION BY SalesOrderNumber)
            *100 AS DECIMAL(5,2)) AS PctByProduct
FROM dbo.FactResellerSales
WHERE SalesOrderNumber IN(N'SO43659',N'SO43664') AND
      ProductKey LIKE '2%'
ORDER BY SalesOrderNumber,ProductKey;

The first start of this result set is:

OrderNumber Product Qty Total PctByProduct

----------- ------- --- ----- ------------

SO43659 218 6 16 37.50

SO43659 220 4 16 25.00

SO43659 223 2 16 12.50

SO43659 229 2 16 18.75

See Also

Reference

Functions (SQL Server PDW)


PARSENAME (SQL Server PDW)

[image]

PARSENAME (SQL Server PDW)
See Also Send Feedback

Returns the specified part of an object name. The parts of an object that can be retrieved are the object name, owner name, database name, and server name.

Note Note

The PARSENAME function does not indicate whether an object by the specified name exists. PARSENAME just returns the specified part of the specified object name.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

PARSENAME ( 'object_name' , object_piece )

Arguments

'object_name'

The name of the object for which to retrieve the specified object part. object_name is sysname. This parameter is an optionally-qualified object name. If all parts of the object name are qualified, this name can have four parts: the server name, the database name, the owner name, and the object name.

object_piece

The object part to return. object_piece is of type int, and can have these values:

1 = Object name

2 = Schema name

3 = Database name

4 = Server name

Return Types

nchar

Remarks

PARSENAME returns NULL if one of the following conditions is true:

  • Either object_name or object_piece is NULL.

  • A syntax error occurs.

  • The requested object part has a length of 0 and is not a valid Microsoft SQL Server PDW identifier.

Examples

The following example uses PARSENAME to return information about the Person table in the AdventureWorks2012 database.

Copy image Copy Code
USE AdventureWorksPDW2012;
SELECT PARSENAME('AdventureWorksPDW2012.dbo.DimCustomer', 1) AS 'Object Name';
SELECT PARSENAME('AdventureWorksPDW2012.dbo.DimCustomer', 2) AS 'Schema Name';
SELECT PARSENAME('AdventureWorksPDW2012.dbo.DimCustomer', 3) AS 'Database Name';
SELECT PARSENAME('AdventureWorksPDW2012.dbo.DimCustomer', 4) AS 'Server Name';
GO

Here is the result set.

Object Name

------------------------------

DimCustomer

(1 row(s) affected)

Schema Name

------------------------------

dbo

(1 row(s) affected)

Database Name

------------------------------

AdventureWorksPDW2012

(1 row(s) affected)

Server Name

------------------------------

(null)

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


PATINDEX (SQL Server PDW)

[image]

PATINDEX (SQL Server PDW)
See Also Send Feedback

Returns the starting position of the first occurrence of a pattern in a specified expression, or zeros if the pattern is not found, on all valid text and character data types in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

PATINDEX ( %pattern% , expression )

Arguments

pattern

A literal string or search pattern. Wildcard characters can be used; however, the % character must be included both before and after pattern, except when searching for starting or ending characters. pattern is an expression of the character string data type category.

For more information on pattern matching, see "Pattern Matching in Search Conditions" in LIKE (SQL Server PDW).

expression

An expression, typically a column that is searched for the specified pattern. expression is of the character string data type category.

Return Types

int

Remarks

If either pattern or expression is NULL, PATINDEX returns NULL.

Examples

A. Using a pattern with PATINDEX

The following example finds the position at which the pattern wheel starts in the EnglishDescription column in the dbo.DimProduct table for every row that contains the characters wheel. If you do not restrict the rows to be searched by using a WHERE clause, the query returns all rows in the table and reports nonzero values for those rows in which the pattern was found and zero for all rows in which the pattern was not found.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT ProductKey,
    PATINDEX('%wheel%',EnglishDescription)AS StartingPosition,
EnglishDescription
FROM dbo.DimProduct
WHERE EnglishDescription LIKE '%wheel%'
ORDER BY ProductKey;

Here is a partial result set.

ProductKey StartingPosition EnglishDescription

---------- ---------------- ----------------------------

410 22 Replacement mountain wheel for entry-level rider.

411 22 Replacement mountain wheel for the casual to serious rider.

412 39 High-performance mountain replacement wheel.

413 24 Replacement road front wheel for entry-level cyclist.

415 8 Strong wheel with double-walled rim.

B. Using wildcard characters with PATINDEX

The following example uses wildcard characters to find the position at which the pattern whe_l starts in the EnglishDescription column in the dbo.DimProduct table, where the underscore is a wildcard representing any character. If you do not restrict the rows to be searched, the query returns all rows in the table and reports nonzero values for those rows in which the pattern was found.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT ProductKey,
    PATINDEX('%whe_l%',EnglishDescription)AS StartingPosition,
EnglishDescription
FROM dbo.DimProduct
WHERE EnglishDescription LIKE '%wheel%'
ORDER BY ProductKey;

See Also

Reference

Expressions (SQL Server PDW)
CHARINDEX (SQL Server PDW)


PERCENTILE_CONT (SQL Server PDW)

[image]

PERCENTILE_CONT (SQL Server PDW)
See Also Send Feedback

PERCENTILE_CONT calculates the percentile value based on a continuous distribution of the column values; the result is interpolated and might not be equal to any of the specific column values.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

PERCENTILE_CONT  ( literal ) WITHIN GROUP
     ( ORDER BY order_by_expression [ASC | DESC ] )
    OVER  ( [ PARTITION BY partition_by_expression ] )

Arguments

literal

The percentile to compute. The literal is between 0.0 and 1.0.

WITHIN GROUP ( ORDER BY order_by_expression [ASC | DESC ])

Specifies a list of numeric values to use as input to the function. The order_by_expression can contain column references. The order_by_expression must evaluate to one of the following data types:

  • Exact numeric (bigint, numeric, bit, smallint, decimal, smallmoney, int, tinyint, money)

  • Approximate numeric (float, real)

For more information, see ORDER BY (SQL Server PDW).

OVER ( [ PARTITION BY partition_by_expression ] )

Determines the partitioning of the rowset before the analytic function is applied. The PERCENTILE_CONT function is used within a SELECT statement, and is applied to the results of the FROM clause. The percentile function is applied to each partition. For more information about the partition_by_expression syntax, see OVER Clause (SQL Server PDW).

Return Types

The return type is always FLOAT(53) and is not determined by the order_by_expression type. The result can be NULL.

General Remarks

PERCENTILE_CONT ignores NULL values in the results of the FROM clause.

Limitations and Restrictions

The PERCENTILE_CONT function is computed by using approximate numerics; running the function on a different ordering of the data, or on different hardware can produce different results.

Examples

A. Basic syntax example

The following example uses PERCENTILE_CONT and PERCENTILE_DISC to find the median employee salary in each department. Note that these functions may not return the same value. This is because PERCENTILE_CONT interpolates the appropriate value, whether or not it exists in the data set, while PERCENTILE_DISC always returns an actual value from the set.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT DepartmentName
,PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY BaseRate)
    OVER (PARTITION BY DepartmentName) AS MedianCont
,PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY BaseRate)
    OVER (PARTITION BY DepartmentName) AS MedianDisc
FROM dbo.DimEmployee;

Here is a partial result set.

DepartmentName MedianCont MedianDisc

-------------------- ---------- ----------

Document Control 16.826900 16.8269

Engineering 34.375000 32.6923

Human Resources 17.427850 16.5865

Shipping and Receiving 9.250000 9.0000

See Also


PERCENTILE_DISC (SQL Server PDW)

[image]

PERCENTILE_DISC (SQL Server PDW)
See Also Send Feedback

PERCENTILE_DISC computes a specific percentile for sorted values in an entire rowset or within distinct partitions of a rowset in SQL Server PDW. For example, PERCENTILE_DISC (0.5) will compute the 50th percentile (that is, the median) of an expression. PERCENTILE_DISC calculates the percentile based on a discrete distribution of the column values; the result is equal to a specific value in the column.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

PERCENTILE_DISC ( literal ) WITHIN GROUP
    ( ORDER BY order_by_expression [ASC | DESC ] )
    OVER ( [ <partition_by_clause> ] )

Arguments

literal

The percentile to compute. The value must range between 0.0 and 1.0.

WITHIN GROUP ( ORDER BY order_by_expression [ASC | DESC ])

Specifies a list of numeric values to sort and compute the percentile over. The order_by_expression can contain column references. The expression must evaluate to a number or a data type that can be converted to a number. For example, order_by_expression can evaluate to date or datetime, but cannot evaluate to type bit, char, or varchar. An integer cannot represent a column in the <order_by_clause>. For more information, see ORDER BY (SQL Server PDW).

OVER

Determines the partitioning of the rowset before the analytic function is applied.

<partition_by_clause>

Divides the result set produced by the FROM clause into partitions to which the percentile function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

Return Types

The return type is determined by the order_by_expression type.

order_by_expression Data Type | Return Value Data Type
bigint | bigint
binary or varbinary | error
bit | error
char or varchar | error
datetime | datetime
datetime2 | datetime2
datetimeoffset | datetimeoffset
decimal | decimal
float or real | float or real
int | int
money | money
smallmoney | smallmoney
smalldatetime | smalldatetime
smallint | smallint
time | time
tinyint | tinyint

General Remarks

Any nulls in the data set are ignored.

Examples

A. Basic syntax example

The following example uses PERCENTILE_CONT and PERCENTILE_DISC to find the median employee salary in each department. Note that these functions may not return the same value. This is because PERCENTILE_CONT interpolates the appropriate value, whether or not it exists in the data set, while PERCENTILE_DISC always returns an actual value from the set.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT DepartmentName
       ,PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY BaseRate)
        OVER (PARTITION BY DepartmentName) AS MedianCont
       ,PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY BaseRate)
        OVER (PARTITION BY DepartmentName) AS MedianDisc
FROM dbo.DimEmployee;

Here is a partial result set.

DepartmentName MedianCont MedianDisc

-------------------- ---------- ----------

Document Control 16.826900 16.8269

Engineering 34.375000 32.6923

Human Resources 17.427850 16.5865

Shipping and Receiving 9.250000 9.0000

See Also


PI (SQL Server PDW)

[image]

PI (SQL Server PDW)
See Also Send Feedback

Returns the constant value of PI.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

PI ( )

Return Types

float

Examples

The following example returns the value of PI.

Copy image Copy Code
SELECT PI();
GO

Here is the result set.

Copy image Copy Code
------------------------
3.14159265358979

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


POWER (SQL Server PDW)

[image]

POWER (SQL Server PDW)
See Also Send Feedback

Returns the exponential value of one expression raised to the power of the second expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

POWER ( float_expression , y )

Arguments

float_expression

An expression of type float or of a type that can be implicitly converted to float, except for the bit data type.

y

The power to which to raise float_expression. y can be an expression of the exact numeric or approximate numeric data type category, except for the bit data type.

Return Types

Same as float_expression.

Examples

The following example shows returns POWER results for 2.0 to the 3rd power.

Copy image Copy Code
SELECT POWER(2.0, 3);

Here is the result set.

------------

8.0

See Also

Reference

SQUARE (SQL Server PDW)
Expressions (SQL Server PDW)
Functions (SQL Server PDW)


QUOTENAME (SQL Server PDW)

[image]

QUOTENAME (SQL Server PDW)
See Also Send Feedback

Returns a Unicode string with the delimiters added to make the input string a valid SQL Server delimited identifier.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

QUOTENAME ( 'character_string' [ , 'quote_character' ] )

Arguments

'character_string'

Is a string of Unicode character data. character_string is sysname and is limited to 128 characters. Inputs greater than 128 characters return NULL.

'quote_character'

Is a one-character string to use as the delimiter. Can be a single quotation mark ( ' ), a left or right bracket ( [ ] ), or a double quotation mark ( " ). If quote_character is not specified, brackets are used.

Return Types

nvarchar(258)

Examples

The following example takes the character string abc def and uses the [ and ] characters to create a valid SQL Server delimited identifier.

Copy image Copy Code
SELECT QUOTENAME('abc def');

Here is the result set.

Copy image Copy Code
[abc def]

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


RADIANS (SQL Server PDW)

[image]

RADIANS (SQL Server PDW)
See Also Send Feedback

Returns radians when a numeric expression, in degrees, is entered.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

RADIANS ( numeric_expression )

Arguments

numeric_expression

Is an expression of the exact numeric or approximate numeric data type category, except for the bit data type.

Return Types

Returns the same type as numeric_expression.

Examples

A. Using RADIANS to show 0.0

The following example returns a result of 0.0 because the numeric expression to convert to radians is too small for the RADIANS function.

Copy image Copy Code
SELECT RADIANS(1e-307)
GO

Here is the result set.

Copy image Copy Code
-------------------
0.0
(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


RANK (SQL Server PDW)

[image]

RANK (SQL Server PDW)
See Also Send Feedback

Returns the rank of each row that is contained in each partition of a result set in SQL Server PDW. The rank of a row is one plus the number of ranks that come before the row in question. Use this analytic function to assign ranking values to a set of rows according to some criterion such as an increasing numeric value.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

RANK ( ) OVER ( [ < partition_by_clause > ] < order_by_clause > )

Arguments

< partition_by_clause >

Divides the result set produced by the FROM clause into partitions to which the RANK function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

< order_by_clause >

Determines the order in which the RANK values are applied to the rows in a partition. For more information, see ORDER BY (SQL Server PDW). An integer cannot represent a column when order_by_clause is used.

Return Types

bigint

Remarks

If two or more rows tie for a rank, each tied rows receives the same rank. For example, if the two top salespeople have the same year-to-date sales value, they are both ranked one. The salesperson with the next highest year-to-date sales value is ranked number three, because there are two rows that are ranked higher. Therefore, the RANK function does not always return consecutive integers. This behavior differs from the DENSE_RANK function. For more information, see DENSE_RANK (SQL Server PDW).

The sort order that is used for the whole query determines the order in which the rows appear in a result set.

Examples

The following example ranks the sales representatives in each sales territory according to their total sales. The rowset is partitioned by SalesTerritoryGroup and sorted by SalesAmountQuota.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT LastName, SUM(SalesAmountQuota) AS TotalSales, SalesTerritoryRegion,
    RANK() OVER (PARTITION BY SalesTerritoryRegion ORDER BY SUM(SalesAmountQuota) DESC ) AS RankResult
FROM dbo.DimEmployee AS e
INNER JOIN dbo.FactSalesQuota AS sq ON e.EmployeeKey = sq.EmployeeKey
INNER JOIN dbo.DimSalesTerritory AS st ON e.SalesTerritoryKey = st.SalesTerritoryKey
WHERE SalesPersonFlag = 1 AND SalesTerritoryRegion != N'NA'
GROUP BY LastName, SalesTerritoryRegion;

Here is the result set.

LastName TotalSales SalesTerritoryGroup RankResult

---------------- ------------- ------------------- --------

Tsoflias 1687000.0000 Australia 1

Saraiva 7098000.0000 Canada 1

Vargas 4365000.0000 Canada 2

Carson 12198000.0000 Central 1

Varkey Chudukatil 5557000.0000 France 1

Valdez 2287000.0000 Germany 1

Blythe 11162000.0000 Northeast 1

Campbell 4025000.0000 Northwest 1

Ansman-Wolfe 3551000.0000 Northwest 2

Mensa-Annan 2753000.0000 Northwest 3

Reiter 8541000.0000 Southeast 1

Mitchell 11786000.0000 Southwest 1

Ito 7804000.0000 Southwest 2

Pak 10514000.0000 United Kingdom 1

See Also

Reference

Functions (SQL Server PDW)
DENSE_RANK (SQL Server PDW)


REPLACE (SQL Server PDW)

[image]

REPLACE (SQL Server PDW)
See Also Send Feedback

Replaces all occurrences of a specified string value with another string value in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

REPLACE ( string_expression , string_pattern , string_replacement )

Arguments

string_expression

The string expression to be searched. string_expression must be a character data type.

string_pattern

The substring to be found. string_pattern must be a character. string_pattern cannot be an empty string ('').

string_replacement

The replacement string. string_replacement must be a character data type.

Return Types

Returns the same type as string_expression.

Returns NULL if any one of the arguments is NULL.

Examples

The following example replaces the string cde in abcdefghi with xxx.

Copy image Copy Code
SELECT REPLACE('abcdefghicde','cde','xxx');

Here is the result set.

--------------------------

abxxxfghixxx

See Also

Reference

Expressions (SQL Server PDW)
CHARINDEX (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


REPLICATE (SQL Server PDW)

[image]

REPLICATE (SQL Server PDW)
See Also Send Feedback

Repeats a string value a specified number of times in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

REPLICATE ( string_expression, integer_expression )

Arguments

string_expression

An expression of a character string. string_expression must be one of the supported character data types.

integer_expression

An expression of any integer type. This expression can be a constant or an int column. If integer_expression is negative, NULL is returned.

Return Types

Returns the same type as string_expression.

Examples

The following example replicates a 0 character four times in front of an ItemCode value.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EnglishProductName AS Name,
   ProductAlternateKey AS ItemCode,
   REPLICATE('0', 4) + ProductAlternateKey AS FullItemCode
FROM dbo.DimProduct
ORDER BY Name;

Here are the first rows in the result set.

Name ItemCode FullItemCode

------------------------ -------------- ---------------

Adjustable Race AR-5381 0000AR-5381

All-Purpose Bike Stand ST-1401 0000ST-1401

AWC Logo Cap CA-1098 0000CA-1098

AWC Logo Cap CA-1098 0000CA-1098

AWC Logo Cap CA-1098 0000CA-1098

BB Ball Bearing BE-2349 0000BE-2349

See Also

Reference

CAST and CONVERT (SQL Server PDW)
Expressions (SQL Server PDW)
RTRIM (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


REVERSE (SQL Server PDW)

[image]

REVERSE (SQL Server PDW)
See Also Send Feedback

Returns the reverse order of a string value.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

REVERSE ( string_expression )

Arguments

string_expression

string_expression is an expression of a string or binary data type. string_expression can be a constant, variable, or column of either character or binary data.

Return Types

varchar or nvarchar

Remarks

string_expression must be of a data type that is implicitly convertible to varchar. Otherwise, use CAST to explicitly convert string_expression.

Supplementary Characters (Surrogate Pairs)

When using SC collations, the REVERSE function will not reverse the order of two halves of a surrogate pair.

Examples

The following example returns names of all databases, and the names with the characters reversed.

Copy image Copy Code
SELECT name, REVERSE(name) FROM sys.databases;
GO

The following example reverses the characters in a variable.

Copy image Copy Code
DECLARE @myvar varchar(10);
SET @myvar = 'sdrawkcaB';
SELECT REVERSE(@myvar) AS Reversed ;
GO

The following example makes an implicit conversion from an int data type into varchar data type and then reverses the result.

Copy image Copy Code
SELECT REVERSE(1234) AS Reversed ;
GO

See Also

Reference

Functions (SQL Server PDW)


RIGHT (SQL Server PDW)

[image]

RIGHT (SQL Server PDW)
See Also Send Feedback

Returns the right part of a character string with the specified number of characters in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

RIGHT ( character_expression , integer_expression )

Arguments

character_expression

An expressions of character data. character_expression can be a constant, variable, or column. character_expression can be of any data type that can be implicitly converted to varchar or nvarchar. Otherwise, use the CAST function to explicitly convert character_expression.

integer_expression

A positive integer that specifies how many characters of the character_expression will be returned. If integer_expression is negative, an error is returned.

Return Types

Returns varchar or nvarchar.

Examples

A. Using RIGHT with a column

The following example returns the five rightmost characters of each last name in the DimEmployee table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT RIGHT(LastName, 5) AS Name
FROM dbo.DimEmployee
ORDER BY EmployeeKey;

Here is a partial result set.

Name

-----

lbert

Brown

rello

lters

B. Using RIGHT with a character string

The following example uses RIGHT to return the two rightmost characters of the character string abcdefg.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP(1) RIGHT('abcdefg',2) FROM dbo.DimProduct;

Here is the result set.

-------

fg

See Also

Reference

Expressions (SQL Server PDW)
LEFT (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


ROUND (SQL Server PDW)

[image]

ROUND (SQL Server PDW)
See Also Send Feedback

Returns a numeric value, rounded to the specified length or precision in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ROUND ( numeric_expression , length )

Arguments

numeric_expression

An expression of the exact numeric or approximate numeric data type category, except for the bit data type.

length

The precision to which numeric_expression is to be rounded. length can be of any numeric type. When length is a positive number, numeric_expression is rounded to the number of decimal positions specified by length. When length is a negative number, numeric_expression is rounded on the left side of the decimal point, as specified by length.

Return Types

Returns the same type as numeric_expression.

Remarks

ROUND always returns a value. If length is negative and larger in absolute value than the number of digits before the decimal point, ROUND returns 0. ROUND returns a rounded numeric_expression, regardless of data type, when length is a negative number.

Examples | Result
ROUND(748.58, -1) | 750.00
ROUND(748.58, -2) | 700.00
ROUND(748.58, -3) | 1000.00
ROUND(748.58, -4) | 0

Examples

A. Using ROUND and estimates

The following example shows two expressions that demonstrate by using ROUND the last digit is always an estimate.

Copy image Copy Code
SELECT ROUND(123.994999, 3), ROUND(123.995444, 3);

Here is the result set.

-------- ---------

123.995000 123.995444

B. Using ROUND and rounding approximations

The following example shows rounding and approximations.

Copy image Copy Code
SELECT ROUND(123.4545, 2), ROUND(123.45, -2);

Here is the result set.

-------- ----------

123.45 100.00

See Also

Reference

Functions (SQL Server PDW)
Expressions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


ROW_NUMBER (SQL Server PDW)

[image]

ROW_NUMBER (SQL Server PDW)
See Also Send Feedback

Returns the sequential number of a row in a partition of a result set, starting at 1 for the first row in each partition in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ROW_NUMBER ( ) OVER ( [ <partition_by_clause> ] <order_by_clause> )

Arguments

<partition_by_clause>

Divides the result set produced by the FROM clause into partitions to which the ROW_NUMBER function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

<order_by_clause>

Determines the order in which the ROW_NUMBER value is assigned to the rows in a partition. For more information, see ORDER BY (SQL Server PDW). An integer cannot represent a column when the <order_by_clause> is used.

Return Types

bigint

Remarks

The ORDER BY clause determines the sequence in which the rows are assigned their unique ROW_NUMBER in a specified partition.

Examples

A. Returning the row number for salespeople

The following example returns the ROW_NUMBER for sales representatives based on their assigned sales quota.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT ROW_NUMBER() OVER(ORDER BY SUM(SalesAmountQuota) DESC) AS RowNumber,
    FirstName, LastName,
    CONVERT(varchar(13), SUM(SalesAmountQuota),1) AS SalesQuota
FROM dbo.DimEmployee AS e
INNER JOIN dbo.FactSalesQuota AS sq
    ON e.EmployeeKey = sq.EmployeeKey
WHERE e.SalesPersonFlag = 1
GROUP BY LastName, FirstName;

Here is a partial result set.

RowNumber FirstName LastName SalesQuota

--------- --------- ------------------ -------------

1 Jillian Carson 12,198,000.00

2 Linda Mitchell 11,786,000.00

3 Michael Blythe 11,162,000.00

4 Jae Pak 10,514,000.00

B. Using ROW_NUMBER() with PARTITION

The following example shows using the ROW_NUMBER function with the PARTITION BY argument. This causes the ROW_NUMBER function to number the rows in each partition.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT ROW_NUMBER() OVER(PARTITION BY SalesTerritoryKey ORDER BY SUM(SalesAmountQuota) DESC) AS RowNumber,
    LastName, SalesTerritoryKey AS Territory,
    CONVERT(varchar(13), SUM(SalesAmountQuota),1) AS SalesQuota
FROM dbo.DimEmployee AS e
INNER JOIN dbo.FactSalesQuota AS sq
    ON e.EmployeeKey = sq.EmployeeKey
WHERE e.SalesPersonFlag = 1
GROUP BY LastName, FirstName, SalesTerritoryKey;

Here is a partial result set.

RowNumber LastName Territory SalesQuota

--------- ------------------ --------- -------------

1 Campbell 1 4,025,000.00

2 Ansman-Wolfe 1 3,551,000.00

3 Mensa-Annan 1 2,275,000.00

1 Blythe 2 11,162,000.00

1 Carson 3 12,198,000.00

1 Mitchell 4 11,786,000.00

2 Ito 4 7,804,000.00

See Also

Reference

Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


RTRIM (SQL Server PDW)

[image]

RTRIM (SQL Server PDW)
See Also Send Feedback

Returns a character string after truncating all trailing blanks in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

RTRIM ( character_expression )

Arguments

character_expression

An expression of character data. character_expression can be a constant, variable, or column of character data.

character_expression must be a data type that is implicitly convertible to varchar or nvarchar. Otherwise, use CAST to explicitly convert character_expression.

Return Types

varchar or nvarchar

Examples

The following example demonstrates how to use RTRIM to remove trailing spaces from a character variable.

Copy image Copy Code
SELECT RTRIM('Four spaces are after the period in this sentence.    ') + 'Next string.';

Here is the result set.

Copy image Copy Code
------------------------------------------------------------------------
Four spaces are after the period in this sentence.Next string.

See Also

Reference

Expressions (SQL Server PDW)
LTRIM (SQL Server PDW)
CAST and CONVERT (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


SCHEMA_ID (SQL Server PDW)

[image]

SCHEMA_ID (SQL Server PDW)
See Also Send Feedback

Returns the schema ID associated with a schema name.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SCHEMA_ID ( [ schema_name ] )

Arguments

Term | Definition
schema_name | The name of the schema. schema_name is a sysname. If schema_name is not specified, SCHEMA_ID will return the ID of the default schema of the caller.

Return Types

int

NULL will be returned if schema_name is not a valid schema.

Remarks

SCHEMA_ID will return IDs of system schemas and user-defined schemas. SCHEMA_ID can be called in a select list, in a WHERE clause, and anywhere an expression is allowed.

Examples

A. Returning the default schema ID of a caller

Copy image Copy Code
SELECT SCHEMA_ID();

B. Returning the schema ID of a named schema

Copy image Copy Code
SELECT SCHEMA_ID('dbo');

See Also

Reference

SCHEMA_NAME (SQL Server PDW)


SCHEMA_NAME (SQL Server PDW)

[image]

SCHEMA_NAME (SQL Server PDW)
See Also Send Feedback

Returns the schema name associated with a schema ID.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SCHEMA_NAME ( [ schema_id ] )

Arguments

Term | Definition
schema_id | The ID of the schema. schema_id is an int. If schema_id is not defined, SCHEMA_NAME will return the name of the default schema of the caller.

Return Types

sysname

Returns NULL when schema_id is not a valid ID.

Remarks

SCHEMA_NAME returns names of system schemas and user-defined schemas. SCHEMA_NAME can be called in a select list, in a WHERE clause, and anywhere an expression is allowed.

Examples

A. Returning the name of the default schema of the caller

Copy image Copy Code
SELECT SCHEMA_NAME();

B. Returning the name of a schema by using an ID

Copy image Copy Code
SELECT SCHEMA_NAME(1);

See Also

Reference

SCHEMA_ID (SQL Server PDW)


SERVERPROPERTY (SQL Server PDW)

[image]

SERVERPROPERTY (SQL Server PDW)
Send Feedback

Returns property information about the server instance.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SERVERPROPERTY ( propertyname )

Arguments

propertyname

An expression that contains the property information to be returned for the server. propertyname can be one of the following values.

Property

Values returned

BuildClrVersion

Version of the Microsoft .NET Framework common language runtime (CLR) that was used while building the instance of SQL Server PDW.

Base data type: nvarchar(128)

Collation

Name of the default collation for the server.

NULL = Input is not valid, or an error.

Base data type: nvarchar(128)

CollationID

ID of the SQL Server PDW collation.

Base data type: int

ComparisonStyle

Windows comparison style of the collation.

Base data type: int

ComputerNamePhysicalNetBIOS

NetBIOS name of the local computer on which the instance of SQL Server PDW is currently running.

For a clustered instance of SQL Server PDW on a failover cluster, this value changes as the instance of SQL Server PDW fails over to other nodes in the failover cluster.

On a stand-alone instance of SQL Server PDW, this value remains constant and returns the same value as the MachineName property.

Note Note

If the instance of SQL Server PDW is in a failover cluster and you want to obtain the name of the failover clustered instance, use the MachineName property.

NULL = Input is not valid, or an error.

Base data type: nvarchar(128)

Edition

Installed product edition of the instance of SQL Server PDW. Use the value of this property to determine the features and the limits, such as maximum number of CPUs that are supported by the installed product. 64-bit versions of the Database Engine append (64-bit) to the version. SQL Server PDW returns Enterprise Edition (64-bit).)

Returns:

'Desktop Engine'

'Developer Edition'

'Enterprise Edition'

'Enterprise Evaluation Edition'

'Personal Edition'(Not available for SQL Server PDW.)

'Standard Edition'

'Express Edition'

'Express Edition with Advanced Services'

'Workgroup Edition'

'Windows Embedded SQL'

Base data type: nvarchar(128)

EditionID

Is an identification number that represents the installed product edition of the instance of SQL Server PDW. Use the value of this property to determine features and limits, such as maximum number of CPUs that are supported by the installed product. SQL Server PDW returns 1804890536.)

-1253826760 = Desktop

-1592396055 = Express

-1534726760 = Standard

1333529388 = Workgroup

1804890536 = Enterprise

-323382091 = Personal

-2117995310 = Developer

610778273 = Enterprise Evaluation

1044790755 = Windows Embedded SQL

4161255391 = Express with Advanced Services

Base data type: int

EngineEdition

Database Engine edition of the instance of SQL Server PDW installed on the server. SQL Server PDW returns 6.

1 = Personal or Desktop Engine (Not available for SQL Server PDW.)

2 = Standard (This is returned for Standard and Workgroup.)

3 = Enterprise (This is returned for Enterprise, Enterprise Evaluation, and Developer.)

4 = Express (This is returned for Express, Express with Advanced Services, and Windows Embedded SQL.)

6 = SQL Server PDW

Base data type: int

InstanceName

Name of the instance to which the user is connected.

SQL Server PDW returns NULL.

Base data type: nvarchar(128)

IsClustered

Server instance is configured in a failover cluster. SQL Server PDW returns 1.

1 = Clustered.

0 = Not Clustered.

NULL = Input is not valid, or an error.

Base data type: int

IsFullTextInstalled

The full-text component is installed with the current instance of SQL Server PDW.

1 = Full-text is installed.

0 = Full-text is not installed.

NULL = Input is not valid, or an error.

Base data type: int

IsIntegratedSecurityOnly

Server is in integrated security mode. SQL Server PDW returns NULL.

1 = Integrated security.

0 = Not integrated security.

NULL = Input is not valid, or an error.

Base data type: int

IsSingleUser

Server is in single-user mode. SQL Server PDW returns 0.

1 = Single user.

0 = Not single user

NULL = Input is not valid, or an error.

Base data type: int

LCID

Windows locale identifier (LCID) of the collation.

Base data type: int

LicenseType

Mode of this instance of SQL Server PDW.

PER_SEAT = Per Seat mode

PER_PROCESSOR = Per-processor mode

DISABLED = Licensing is disabled.

Base data type: nvarchar(128)

MachineName

Returns the appliance name, same as ServerName below.

NULL = Input is not valid, or an error.

Base data type: nvarchar(128)

ProductVersion

Version of the instance of SQL Server PDW, in the form of 'major.minor.build'.

Base data type: nvarchar(128)

ProductLevel

Level of the version of the instance of SQL Server PDW.

Returns one of the following:

'RTM' = Original release version

'SPn' = Service pack version

'CTP' = Community Technology Preview version

'AUn' = Appliance Update

Base data type: nvarchar(128)

ResourceLastUpdateDateTime

Returns the date and time that the Resource database was last updated.

Base data type: datetime

ResourceVersion

Returns the version Resource database.

Base data type: nvarchar(128)

ServerName

Returns the appliance name.

NULL = Input is not valid, or an error.

Base data type: nvarchar(128)

SqlCharSet

The SQL character set ID from the collation ID. Always the value of the control node.

Base data type: tinyint

SqlCharSetName

The SQL character set name from the collation. Always the value of the control node.

Base data type: nvarchar(128)

SqlSortOrder

The SQL sort order ID from the collation. Always the value of the control node.

Base data type: tinyint

SqlSortOrderName

The SQL sort order name from the collation. Always the value of the control node.

Base data type: nvarchar(128)

Return Types

sql_variant

Remarks

ServerName Property

The ServerName property provides the Windows server and instance name that together make up the unique server instance

Version Properties

The SERVERPROPERTY function returns individual properties that relate to the version

Examples

The following example uses the SERVERPROPERTY function in a SELECT statement to return information about the current server. This scenario is useful when there are multiple instances of SQL Server installed on a Windows server, and the client must open another connection to the same instance used by the current connection.

Copy image Copy Code
SELECT CONVERT(sysname, SERVERPROPERTY('servername'));

The following example uses the SERVERPROPERTY function in a SELECT statement to return version information about the product.

Copy image Copy Code
SELECT
SERVERPROPERTY('ProductVersion') AS ProductVersion,
SERVERPROPERTY('ProductLevel') AS ProductLevel,
SERVERPROPERTY('Edition') AS Edition,
SERVERPROPERTY('EngineEdition') AS EngineEdition;


SESSION_ID (SQL Server PDW)

[image]

SESSION_ID (SQL Server PDW)
See Also Send Feedback

Returns the ID of the current SQL Server PDW session.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SESSION_ID ( )

Return Value

Returns an nvarchar(32) value.

General Remarks

The session ID is assigned to each user connection when the connection is made. It persists for the duration of the connection. When the connection ends, the session ID is released.

The session ID begins with the alphabetical characters 'SID'. These are case-sensitive and must be capitalized when session ID is used in SQL commands.

You can query the view sys.dm_pdw_exec_sessions to retrieve the same information as this function.

Examples

The following example returns the current session ID.

Copy image Copy Code
SELECT SESSION_ID();

See Also

Reference

DB_NAME (SQL Server PDW)
VERSION (SQLServer PDW)


SESSION_USER (SQL Server PDW)

[image]

SESSION_USER (SQL Server PDW)
Send Feedback

SESSION_USER returns the user name of the current context in the current database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SESSION_USER

Return Types

nvarchar(128)

Remarks

This function takes no arguments. SESSION_USER can be used in queries.

Examples

The following example returns the session user for the current session.

Copy image Copy Code
SELECT SESSION_USER;


SIGN (SQL Server PDW)

[image]

SIGN (SQL Server PDW)
See Also Send Feedback

Returns the positive (+1), zero (0), or negative (-1) sign of the specified expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SIGN ( numeric_expression )

Arguments

numeric_expression

An expression of the exact numeric or approximate numeric data type categories, except for the bit data type.

Return Types

Specified expression | Return type
bigint | bigint
int/smallint/tinyint | int
decimal | decimal
Other types | float

Examples

The following example returns the SIGN values of three numbers.

Copy image Copy Code
SELECT SIGN(-125), SIGN(0), SIGN(564);

Here is the result set.

----- ----- -----

-1 0 1

See Also

Reference

Expressions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


SIN (SQL Server PDW)

[image]

SIN (SQL Server PDW)
See Also Send Feedback

Returns the trigonometric sine of the specified angle, in radians, and in an approximate numeric (float) expression SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SIN ( float_expression )

Arguments

float_expression

An expression of type float or of a type that can be implicitly converted to float.

Return Types

float

Examples

The following example calculates the sine for a specified angle.

Copy image Copy Code
SELECT SIN(45.175643);

Here is the result set.

---------

0.929607

See Also

Reference

Expressions (SQL Server PDW)
ASIN (SQL Server PDW)
Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


SMALLDATETIMEFROMPARTS (SQL Server PDW)

[image]

SMALLDATETIMEFROMPARTS (SQL Server PDW)
See Also Send Feedback

Returns a smalldatetime value for the specified date and time.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SMALLDATETIMEFROMPARTS ( year, month, day, hour, minute )

Arguments

year

Integer expression specifying a year.

month

Integer expression specifying a month.

day

Integer expression specifying a day.

hour

Integer expression specifying hours.

minute

Integer expression specifying minutes.

Return Types

smalldatetime

Remarks

This functions acts like a constructor for a fully initialized smalldatetime value. If the arguments are not valid, then an error is thrown. If required arguments are null, then null is returned.

Examples

Copy image Copy Code
SELECT SMALLDATETIMEFROMPARTS ( 2010, 12, 31, 23, 59 ) AS Result

Here is the result set.

Copy image Copy Code
Result
---------------------------
2011-01-01 00:00:00

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


SOUNDEX (SQL Server PDW)

[image]

SOUNDEX (SQL Server PDW)
See Also Send Feedback

Returns a four-character (SOUNDEX) code to evaluate the similarity of two strings.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SOUNDEX ( character_expression )

Arguments

character_expression

Is an alphanumeric expression of character data. character_expression can be a constant, variable, or column.

Return Types

varchar

Remarks

SOUNDEX converts an alphanumeric string to a four-character code that is based on how the string sounds when spoken. The first character of the code is the first character of character_expression, converted to upper case. The second through fourth characters of the code are numbers that represent the letters in the expression. The letters A, E, I, O, U, H, W, and Y are ignored unless they are the first letter of the string. Zeroes are added at the end if necessary to produce a four-character code. For more information about the SOUNDEX code, see The Soundex Indexing System.

SOUNDEX codes from different strings can be compared to see how similar the strings sound when spoken. The DIFFERENCE function performs a SOUNDEX on two strings, and returns an integer that represents how similar the SOUNDEX codes are for those strings.

SOUNDEX is collation sensitive. String functions can be nested.

Examples

The following example shows the SOUNDEX function and the related DIFFERENCE function. In the first example, the standard SOUNDEX values are returned for all consonants. Returning the SOUNDEX for Smith and Smythe returns the same SOUNDEX result because all vowels, the letter y, doubled letters, and the letter h, are not included.

Copy image Copy Code
-- Using SOUNDEX
SELECT SOUNDEX ('Smith'), SOUNDEX ('Smythe');

Here is the result set. Valid for a Latin1_General collation.

Copy image Copy Code
----- -----
S530  S530

(1 row(s) affected)

The DIFFERENCE function compares the difference of the SOUNDEX pattern results. The following example shows two strings that differ only in vowels. The difference returned is 4, the lowest possible difference.

Copy image Copy Code
-- Using DIFFERENCE
SELECT DIFFERENCE('Smithers', 'Smythers');
GO

Here is the result set. Valid for a Latin1_General collation.

Copy image Copy Code
-----------
4

(1 row(s) affected)

In the following example, the strings differ in consonants; therefore, the difference returned is 2, the greater difference.

Copy image Copy Code
SELECT DIFFERENCE('Anothers', 'Brothers');
GO

Here is the result set. Valid for a Latin1_General collation.

Copy image Copy Code
-----------
2

(1 row(s) affected)

See Also

Reference

DIFFERENCE (SQL Server PDW)
Functions (SQL Server PDW)


SPACE (SQL Server PDW)

[image]

SPACE (SQL Server PDW)
See Also Send Feedback

Returns a string of repeated spaces.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SPACE ( integer_expression )

Arguments

integer_expression

Is a positive integer that indicates the number of spaces. If integer_expression is negative, a null string is returned.

Return Types

varchar

Remarks

To include spaces in Unicode data, or to return more than 8000 character spaces, use REPLICATE instead of SPACE.

Examples

The following example trims the last names and concatenates a comma, two spaces, and the first names of people listed in the DimCustomer table in AdventureWorksPDW2012.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
SELECT RTRIM(LastName) + ',' + SPACE(2) +  LTRIM(FirstName)
FROM dbo.DimCustomer
ORDER BY LastName, FirstName;
GO

See Also

Reference

REPLICATE (SQL Server PDW)
Functions (SQL Server PDW)


SQL_VARIANT_PROPERTY (SQL Server PDW)

[image]

SQL_VARIANT_PROPERTY (SQL Server PDW)
See Also Send Feedback

Returns the base data type and other information about a sql_variant value.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SQL_VARIANT_PROPERTY ( expression , property )

Arguments

expression

Is an expression of type sql_variant.

property

Contains the name of the sql_variant property for which information is to be provided. property is varchar(128), and can be any one of the following values.

Value

Description

Base type of sql_variant returned

BaseType

SQL Server data type, such as:

bigint

binary

char

date

datetime

datetime2

datetimeoffset

decimal

float

int

money

nchar

numeric

nvarchar

real

smalldatetime

smallint

smallmoney

time

tinyint

uniqueidentifier

varbinary

varchar

sysname

NULL = Input is not valid.

Precision

Number of digits of the numeric base data type:

datetime = 23

smalldatetime = 16

float = 53

real = 24

decimal (p,s) and numeric (p,s) = p

money = 19

smallmoney = 10

bigint = 19

int = 10

smallint = 5

tinyint = 3

bit = 1

All other types = 0

int

NULL = Input is not valid.

Scale

Number of digits to the right of the decimal point of the numeric base data type:

decimal (p,s) and numeric (p,s) = s

money and smallmoney = 4

datetime = 3

all other types = 0

int

NULL = Input is not valid.

TotalBytes

Number of bytes required to hold both the metadata and data of the value. This information would be useful in checking the maximum side of data in a sql_variant column. If the value is larger than 900, index creation will fail.

int

NULL = Input is not valid.

Collation

Represents the collation of the particular sql_variant value.

sysname

NULL = Input is not valid.

MaxLength

Maximum data type length, in bytes. For example, MaxLength of nvarchar(50) is 100, MaxLength of int is 4.

int

NULL = Input is not valid.

Return Types

sql_variant

Examples

The following example retrieves SQL_VARIANT_PROPERTY information about a variable named @v1.

Copy image Copy Code
DECLARE @v1 sql_variant;
SET @v1 = 'ABC';
SELECT @v1;
SELECT SQL_VARIANT_PROPERTY(@v1, 'BaseType');
SELECT SQL_VARIANT_PROPERTY(@v1, 'MaxLength');

See Also

Reference

DECLARE (SQL Server PDW)
Functions (SQL Server PDW)


SQRT (SQL Server PDW)

[image]

SQRT (SQL Server PDW)
See Also Send Feedback

Returns the square root of the specified float value in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SQRT ( float_expression )

Arguments

float_expression

An expression of type float or of a type that can be implicitly converted to float. Otherwise, use CAST to explicitly convert expression.

Return Types

float

Examples

The following example returns the square root of numbers 1.00 and 10.00.

Copy image Copy Code
SELECT SQRT(1.00), SQRT(10.00);

Here is the result set.

---------- ------------

1.00 3.16

See Also

Reference

Expressions (SQL Server PDW)
CAST and CONVERT (SQL Server PDW)
EXP (SQL Server PDW)
Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


SQUARE (SQL Server PDW)

[image]

SQUARE (SQL Server PDW)
See Also Send Feedback

Returns the square of the specified float value in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SQUARE ( float_expression )

Arguments

float_expression

An expression of type float or of a type that can be implicitly converted to float.

Return Types

float

Examples

The following example returns the square of each value in the volume column in the containers table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

CREATE TABLE Containers (
    ID int NOT NULL,
    Name varchar(20),
    Volume float(24));

INSERT INTO Containers VALUES (1, 'Cylinder', '125.22');
INSERT INTO Containers VALUES (2, 'Cube', '23.98');

SELECT Name, SQUARE(Volume) AS VolSquared
FROM Containers;

Here is the result set.

Name VolSquared

------------- ----------

Cylinder 15680.05

Cube 575.04

See Also

Reference

POWER (SQL Server PDW)
Functions (SQL Server PDW)


STATS_DATE (PDW)

[image]

STATS_DATE (PDW)
See Also Send Feedback

Returns the date of the most recent update for query optimization statistics in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

STATS_DATE ( object_id , stats_id )

Arguments

object_id

ID of the table with the statistics.

stats_id

ID of the statistics object.

Return Types

Returns datetime on success. Returns NULL on error.

Permissions

Requires membership in the db_owner fixed database role or permission to view the metadata for the table.

General Remarks

The STATS_DATE function, and other system functions, can be used in a SELECT list, a WHERE clause, and anywhere an expression can be used.

Limitations and Restrictions

The results include statistics stored in the Shell database at the Control node level. They do not show the date for statistics that are auto-created by SQL Server on the Compute nodes. For more information, see Understanding Query Optimization Statistics (SQL Server PDW).

Examples

A. Learn when a named statistics was last updated

The following example creates statistics on the LastName column of the DimCustomer table. It then runs a query to show the date of the statistics. Then it udpates the statistics and runs the query again to show the updated date.

Copy image Copy Code
--First, create a statistics object
USE AdventureWorksPDW2012;
GO
CREATE STATISTICS Customer_LastName_Stats
ON AdventureWorksPDW2012.dbo.DimCustomer (LastName)
WITH SAMPLE 50 PERCENT;
GO

--Return the date when Customer_LastName_Stats was last updated
USE AdventureWorksPDW2012;
GO
SELECT stats_id, name AS stats_name,
    STATS_DATE(object_id, stats_id) AS statistics_date
FROM sys.stats s
WHERE s.object_id = OBJECT_ID('dbo.DimCustomer')
    AND s.name = 'Customer_LastName_Stats';
GO

--Update Customer_LastName_Stats so it will have a different timestamp in the next query
GO
UPDATE STATISTICS dbo.dimCustomer (Customer_LastName_Stats);

--Return the date when Customer_LastName_Stats was last updated.
SELECT stats_id, name AS stats_name,
    STATS_DATE(object_id, stats_id) AS statistics_date
FROM sys.stats s
WHERE s.object_id = OBJECT_ID('dbo.DimCustomer')
    AND s.name = 'Customer_LastName_Stats';
GO

B. View the date of the last update for all statistics on a table

This example returns the date for when each statistics object on the DimCustomer table was last updated.

Copy image Copy Code
--Return the dates all statistics on the table were last updated.
SELECT stats_id, name AS stats_name,
    STATS_DATE(object_id, stats_id) AS statistics_date
FROM sys.stats s
WHERE s.object_id = OBJECT_ID('dbo.DimCustomer');
GO

If statistics correspond to an index, the stats_id value in the sys.stats catalog view is the same as the index_id value in the sys.indexes catalog view, and the following query returns the same results as the preceding query. If statistics do not correspond to an index, they are in the sys.stats results but not in the sys.indexes results.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
SELECT name AS index_name,
    STATS_DATE(object_id, index_id) AS statistics_update_date
FROM sys.indexes
WHERE object_id = OBJECT_ID('dbo.DimCustomer');
GO

See Also

Reference

CREATE STATISTICS (SQL Server PDW)

Concepts

Understanding Query Optimization Statistics (SQL Server PDW)


STDEV (SQL Server PDW)

[image]

STDEV (SQL Server PDW)
See Also Send Feedback

Returns the statistical standard deviation of all values in the specified expression in a partition or table in SQL Server PDW. Use this function in the select list or HAVING clause of a SELECT statement.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Aggregate Function Syntax
STDEV ( [ ALL | DISTINCT ] expression )
-- Analytic Function Syntax
STDEV (expression) OVER ( [ partition_by_clause ] order_by_clause  )

Arguments

ALL

Applies the aggregate function to all values. ALL is the default.

DISTINCT

Specifies that each unique value is considered.

expression

An expression of the exact numeric or approximate numeric data type category, except for the bit data type. Aggregate functions, analytic functions, and subqueries are not permitted.

partition_by_clause

Divides the result set produced by the FROM (SQL Server PDW) clause into partitions to which the STDEV function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

order_by_clause

Determines the order in which the STDEV values are applied to the rows in a partition. An integer cannot represent a column in order_by_clause. When the order_by_clause is absent, all rows in the same partition are assigned the same value for the STDEV() OVER computation. When the order_by_clause is present, the STDEV() OVER computation gives you a “cumulative stdev” in the partition. Within a partition, rows with identical values for all elements of the order_by_clause will return identical results for the STDEV() OVER computation. For more information, see ORDER BY (SQL Server PDW).

Return Types

float

General Remarks

If STDEV is used on all items in a SELECT statement, each value in the result set is included in the calculation. STDEV can be used with numeric columns only. Null values are ignored.

Limitations and Restrictions

The STDEV() OVER analytic function does not support the use of the ALL or DISTINCT modifiers.

Examples

Using STDEV

The following example returns the standard deviation of the sales quota values in the table dbo.FactSalesQuota. The first column contains the standard deviation of all distinct values and the second column contains the standard deviation of all values including any duplicates values.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT STDEV(DISTINCT SalesAmountQuota)AS Distinct_Values, STDEV(SalesAmountQuota) AS All_Values
FROM dbo.FactSalesQuota;

Here is the result set.

Distinct_Values All_Values

---------------- ----------------

398974.27 398450.57

B. Using STDEV with OVER

The following example returns the standard deviation of the sales quota values for each quarter in a calendar year. Notice that the ORDER BY in the OVER clause orders the STDEV and the ORDER BY of the SELECT statement orders the result set.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT CalendarYear AS Year, CalendarQuarter AS Quarter, SalesAmountQuota AS SalesQuota,
       STDEV(SalesAmountQuota) OVER (ORDER BY CalendarYear, CalendarQuarter) AS StdDeviation
FROM dbo.FactSalesQuota
WHERE EmployeeKey = 272 AND CalendarYear = 2002
ORDER BY CalendarQuarter;

Here is the result set.

Year Quarter SalesQuota StdDeviation

---- ------- ---------------------- -------------------

2002 1 91000.0000 null

2002 2 140000.0000 34648.23

2002 3 70000.0000 35921.21

2002 4 154000.0000 39752.36

See Also

Reference

Functions (SQL Server PDW)
Expressions (SQL Server PDW)
STDEVP (SQL Server PDW)


STDEVP (SQL Server PDW)

[image]

STDEVP (SQL Server PDW)
See Also Send Feedback

Returns the statistical standard deviation for the population for all values within the specified expression in a table or partition in SQL Server PDW. Use this function in the select list or HAVING clause of a SELECT statement.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Aggregate Function Syntax
STDEVP ( [ ALL | DISTINCT ] expression )
-- Analytic Function Syntax
STDEVP (expression) OVER ( [ partition_by_clause ] order_by_clause  )

Arguments

ALL

Applies the aggregate function to all values. ALL is the default.

DISTINCT

Specifies that each unique value is considered.

expression

An expression of the exact numeric or approximate numeric data type category, except for the bit data type. Aggregate functions, analytic functions, and subqueries are not permitted.

partition_by_clause

Divides the result set produced by the FROM (SQL Server PDW) clause into partitions to which the STDEVP function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

order_by_clause

Determines the order in which the STDEVP values are applied to the rows in a partition. An integer cannot represent a column in order_by_clause. When the order_by_clause is absent, all rows in the same partition are assigned the same value for the STDEVP() OVER computation. When the order_by_clause is present, the STDEVP() OVER computation gives you a “cumulative stdevp” in the partition. Within a partition, rows with identical values for all elements of the order_by_clause will return identical results for the STDEVP() OVER computation. For more information, see ORDER BY (SQL Server PDW).

Return Types

float

General Remarks

If STDEVP is used on all items in a SELECT statement, each value in the result set is included in the calculation. STDEVP can be used with numeric columns only. Null values are ignored.

Limitations and Restrictions

The STDEVP() OVER analytic function does not support the use of the ALL or DISTINCT modifiers.

Examples

Using STDEVP

The following example returns the STDEVP of the sales quota values in the table dbo.FactSalesQuota. The first column contains the standard deviation of all distinct values and the second column contains the standard deviation of all values including any duplicates values.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT STDEVP(DISTINCT SalesAmountQuota)AS Distinct_Values, STDEVP(SalesAmountQuota) AS All_Values
FROM dbo.FactSalesQuota;SELECT STDEVP(DISTINCT Quantity)AS Distinct_Values, STDEVP(Quantity) AS All_Values
FROM ProductInventory;

Here is the result set.

Distinct_Values All_Values

---------------- ----------------

397676.79 397226.44

B. Using STDEVP with OVER

The following example returns the STDEVP of the sales quota values for each quarter in a calendar year. Notice that the ORDER BY in the OVER clause orders the STDEVP and the ORDER BY of the SELECT statement orders the result set.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT CalendarYear AS Year, CalendarQuarter AS Quarter, SalesAmountQuota AS SalesQuota,
       STDEVP(SalesAmountQuota) OVER (ORDER BY CalendarYear, CalendarQuarter) AS StdDeviation
FROM dbo.FactSalesQuota
WHERE EmployeeKey = 272 AND CalendarYear = 2002
ORDER BY CalendarQuarter;

Here is the result set.

Year Quarter SalesQuota StdDeviation

---- ------- ---------------------- -------------------

2002 1 91000.0000 0.00

2002 2 140000.0000 24500.00

2002 3 70000.0000 29329.55

2002 4 154000.0000 34426.55

See Also

Reference

Functions (SQL Server PDW)
Expressions (SQL Server PDW)
STDEV (SQL Server PDW)


STR (SQL Server PDW)

[image]

STR (SQL Server PDW)
See Also Send Feedback

Returns character data converted from numeric data.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

STR ( float_expression [ , length [ , decimal ] ] )

Arguments

float_expression

Is an expression of approximate numeric (float) data type with a decimal point.

length

Is the total length. This includes decimal point, sign, digits, and spaces. The default is 10.

decimal

Is the number of places to the right of the decimal point. decimal must be less than or equal to 16. If decimal is more than 16 then the result is truncated to sixteen places to the right of the decimal point.

Return Types

varchar

Remarks

If supplied, the values for length and decimal parameters to STR should be positive. The number is rounded to an integer by default or if the decimal parameter is 0. The specified length should be greater than or equal to the part of the number before the decimal point plus the number's sign (if any). A short float_expression is right-justified in the specified length, and a long float_expression is truncated to the specified number of decimal places. For example, STR(12,10) yields the result of 12. This is right-justified in the result set. However, STR(1223,2) truncates the result set to **. String functions can be nested.

Note Note

To convert to Unicode data, use STR inside a CONVERT or CAST conversion function. For more information, see CAST and CONVERT (SQL Server PDW).

Examples

The following example converts an expression that is made up of five digits and a decimal point to a six-position character string. The fractional part of the number is rounded to one decimal place.

Copy image Copy Code
SELECT STR(123.45, 6, 1);
GO

Here is the result set.

Copy image Copy Code
------
 123.5

(1 row(s) affected)

When the expression exceeds the specified length, the string returns ** for the specified length.

Copy image Copy Code
SELECT STR(123.45, 2, 2);
GO

Here is the result set.

Copy image Copy Code
--
**

(1 row(s) affected)

Even when numeric data is nested within STR, the result is character data with the specified format.

Copy image Copy Code
SELECT STR (FLOOR (123.45), 8, 3);
GO

Here is the result set.

Copy image Copy Code
--------
 123.000

(1 row(s) affected)

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)

Other Resources

String Functions (Transact-SQL)


STUFF (SQL Server PDW)

[image]

STUFF (SQL Server PDW)
See Also Send Feedback

The STUFF function inserts a string into another string. It deletes a specified length of characters in the first string at the start position and then inserts the second string into the first string at the start position.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

STUFF ( character_expression , start , length , replaceWith_expression )

Arguments

character_expression

Is an expression of character data. character_expression can be a constant, variable, or column of either character or binary data.

start

Is an integer value that specifies the location to start deletion and insertion. If start or length is negative, a null string is returned. If start is longer than the first character_expression, a null string is returned. start can be of type bigint.

length

Is an integer that specifies the number of characters to delete. If length is longer than the first character_expression, deletion occurs up to the last character in the last character_expression. length can be of type bigint.

replaceWith_expression

Is an expression of character data. character_expression can be a constant, variable, or column of either character or binary data. This expression will replace length characters of character_expression beginning at start.

Return Types

Returns character data if character_expression is one of the supported character data types. Returns binary data if character_expression is one of the supported binary data types.

Remarks

If the start position or the length is negative, or if the starting position is larger than length of the first string, a null string is returned. If the start position is 0, a null value is returned. If the length to delete is longer than the first string, it is deleted to the first character in the first string.

An error is raised if the resulting value is larger than the maximum supported by the return type.

Supplementary Characters (Surrogate Pairs)

When using SC collations, both character_expression and replaceWith_expression can include surrogate pairs. The length parameter will count each surrogate in character_expression as a single character.

Examples

The following example returns a character string created by deleting three characters from the first string, abcdef, starting at position 2, at b, and inserting the second string at the deletion point.

Copy image Copy Code
SELECT STUFF('abcdef', 2, 3, 'ijklmn');
GO

Here is the result set.

Copy image Copy Code
---------
aijklmnef

(1 row(s) affected)

See Also

Reference

Functions (SQL Server PDW)


SUBSTRING (SQL Server PDW)

[image]

SUBSTRING (SQL Server PDW)
See Also Send Feedback

Returns part of a character or binary expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SUBSTRING ( value_expression , start_expression ,
        length_expression )

Arguments

value_expression

A character expression.

start_expression

An integer or bigint expression that specifies where the returned characters start. If start_expression is less than 1, the returned expression will begin at the first character that is specified in value_expression. In this case, the number of characters that are returned is the larger of either the sum of start_expression and length_expression, or 0. If start_expression is greater than the number of characters in the value expression, a zero-length expression is returned.

length_expression

A positive integer or bigint expression that specifies how many characters of the value_expression will be returned. If length_expression is negative, an error is generated and the statement is terminated. If the sum of start_expression and length_expression is greater than the number of characters in value_expression, the whole value expression beginning at start_expression is returned.

Return Types

Returns character data if expression is one of the supported character data types. The returned string is the same type as the specified expression with the exceptions shown in the table.

Specified expression | Return type
char/varchar | varchar
nchar/nvarchar | nvarchar

Remarks

The values for start_expression and length_expression must be specified in number of characters for char, varchar, nchar, or nvarchar data types.

Examples

A. Using SUBSTRING with a character string

The following example shows how to return only a part of a character string. From the dbo.DimEmployee table, this query returns the last name in one column with only the first initial in the second column.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT LastName, SUBSTRING(FirstName, 1, 1) AS Initial
FROM dbo.DimEmployee
WHERE LastName LIKE 'Bar%'
ORDER BY LastName;

Here is the result set.

LastName Initial

-------------------- -------

Barbariol A

Barber D

Barreto de Mattos P

The following example shows how to return the second, third, and fourth characters of the string constant abcdef.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT TOP 1 SUBSTRING('abcdef', 2, 3) AS x FROM dbo.DimCustomer;

Here is the result set.

x

-----

bcd

See Also

Reference

Expressions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


SUM (SQL Server PDW)

[image]

SUM (SQL Server PDW)
See Also Send Feedback

Returns the sum of all the values, or only the DISTINCT values, in the expression. SUM can be used with numeric columns only. Null values are ignored. May be followed by the OVER Clause (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SUM ( [ ALL | DISTINCT ] expression )

Arguments

ALL

Applies the aggregate function to all values. ALL is the default.

DISTINCT

Specifies that SUM return the sum of unique values.

expression

A constant or column, and any combination of arithmetic and string operators. expression is an expression of the exact numeric or approximate numeric data type category, except for the bit data type. Aggregate functions and subqueries are not permitted. For more information, see Expressions (SQL Server PDW).

Return Types

Returns the sum of all expression values in the most precise expression data type.

Expression result | Return type
tinyint | int
smallint | int
int | int
bigint | bigint
decimal(p, s) | decimal(38, s)
float | float

Examples

A. A simple SUM example

The following example returns the total number of each product sold in the year 2003.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT ProductKey, SUM(SalesAmount) AS TotalPerProduct
FROM dbo.FactInternetSales
WHERE OrderDateKey >= '20030101'
      AND OrderDateKey < '20040101'
GROUP BY ProductKey
ORDER BY ProductKey;

Here is a partial result set.

ProductKey TotalPerProduct

---------- ---------------

214 31421.0200

217 31176.0900

222 29986.4300

225 7956.1500

B. Calculating group totals with more than one column

The following example calculates the sum of the ListPrice and StandardCost for each color listed in the Product table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT Color, SUM(ListPrice)AS TotalList,
       SUM(StandardCost) AS TotalCost
FROM dbo.DimProduct
GROUP BY Color
ORDER BY Color;

The first part of the result set is shown below:

Color TotalList TotalCost

---------- ------------- --------------

Black 101295.7191 57490.5378

Blue 24082.9484 14772.0524

Grey 125.0000 51.5625

Multi 880.7468 526.4095

NA 3162.3564 1360.6185

See Also

Reference

Expressions (SQL Server PDW)
Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


SUSER_SNAME (SQL Server PDW)

[image]

SUSER_SNAME (SQL Server PDW)
See Also Send Feedback

Returns the current login name or the login name associated with a security identification number (SID). For more information, see the SUSER_SNAME (Transact-SQL) documentation on MSDN. All features in the MSDN documentation may not be available in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SUSER_SNAME ( [ server_user_sid ] )

Arguments

server_user_sid

Is the login security identification number. server_user_sid, which is optional, is varbinary(85). server_user_sid can be the security identification number of any SQL Server login. If server_user_sid is not specified, information about the current user is returned. If the parameter contains the word NULL will return NULL.

Return Types

nvarchar(128)

Remarks

When called without an argument, SUSER_SNAME returns the name of the current security context. When called without an argument within a batch that has switched context by using EXECUTE AS, SUSER_SNAME returns the name of the impersonated context. When called from an impersonated context, ORIGINAL_LOGIN returns the name of the original context.

The SYSTEM_USER and SUSER_NAME() functions are not supported in SQL Server PDW. Use SUSER_SNAME() instead.

Examples

A. Using SUSER_SNAME

The following example returns the login name for the security identification number with a value of 0x01.

Copy image Copy Code
SELECT SUSER_SNAME(0x01);
GO

B. Returning the Current Login

The following example returns the login name of the current login.

Copy image Copy Code
SELECT SUSER_SNAME() AS CurrentLogin;
GO

See Also

Reference

sys.server_principals (SQL Server PDW)
sys.server_permissions (SQL Server PDW)


SWITCHOFFSET (SQL Server PDW)

[image]

SWITCHOFFSET (SQL Server PDW)
See Also Send Feedback

Returns a datetimeoffset value that is changed from the stored time zone offset to a specified new time zone offset.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SWITCHOFFSET ( DATETIMEOFFSET, time_zone )

Arguments

DATETIMEOFFSET

Is an expression that can be resolved to a datetimeoffset(n) value.

time_zone

Is a character string in the format [+|-]TZH:TZM or a signed integer (of minutes) that represents the time zone offset, and is assumed to be daylight-saving aware and adjusted.

Return Type

datetimeoffset with the fractional precision of the DATETIMEOFFSET argument.

Remarks

Use SWITCHOFFSET to select a datetimeoffset value into a time zone offset that is different from the time zone offset that was originally stored. SWITCHOFFSET does not update the stored time_zone value.

SWITCHOFFSET can be used to update a datetimeoffset column.

Using SWITCHOFFSET with the function GETDATE() can cause the query to run slowly. This is because the query optimizer is unable to obtain accurate cardinality estimates for the datetime value.

Examples

The following example uses SWITCHOFFSET to display a different time zone offset than the value stored in the database.

Copy image Copy Code
CREATE TABLE dbo.test
    (
    ColDatetimeoffset datetimeoffset
    );
GO
INSERT INTO dbo.test
VALUES ('1998-09-20 7:45:50.71345 -5:00');
GO
SELECT SWITCHOFFSET (ColDatetimeoffset, '-08:00')
FROM dbo.test;
GO
--Returns: 1998-09-20 04:45:50.7134500 -08:00
SELECT ColDatetimeoffset
FROM dbo.test;
--Returns: 1998-09-20 07:45:50.7134500 -05:00

See Also

Reference

Functions (SQL Server PDW)


SYSDATETIME (SQL Server PDW)

[image]

SYSDATETIME (SQL Server PDW)
See Also Send Feedback

Returns a datetime value that contains the current appliance date and time from the Control Node in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SYSDATETIME ( )

Return Type

datetime2(7)

General Remarks

SQL statements can refer to SYSDATETIME anywhere they can refer to a datetime expression.

Limitations and Restrictions

SYSDATETIME is a nondeterministic function. Views and expressions that reference this function in a column cannot be indexed.

Examples

Copy image Copy Code
SELECT SYSDATETIME();

Here is the result set.

--------------------------

7/20/2013 2:49:59 PM

See Also

Reference

Functions (SQL Server PDW)


SYSDATETIMEOFFSET (SQL Server PDW)

[image]

SYSDATETIMEOFFSET (SQL Server PDW)
See Also Send Feedback

Returns a datetimeoffset(7) value that contains the date and time of the APS. The time zone offset is included.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SYSDATETIMEOFFSET ( )

Return Type

datetimeoffset(7)

Remarks

Transact-SQL statements can refer to SYSDATETIMEOFFSET anywhere they can refer to a datetimeoffset expression.

SYSDATETIMEOFFSET is a nondeterministic function. Views and expressions that reference this function in a column cannot be indexed.

Note Note

APS obtains the date and time values by using the GetSystemTimeAsFileTime() Windows API. The accuracy depends on the computer hardware and version of Windows on which the instance of SQL Server is running. The precision of this API is fixed at 100 nanoseconds. The accuracy can be determined by using the GetSystemTimeAdjustment() Windows API.

Examples

The following examples use the six SQL Server system functions that return current date and time to return the date, time, or both. The values are returned in series; therefore, their fractional seconds might be different.

A. Showing the formats that are returned by the date and time functions

The following example shows the different formats that are returned by the date and time functions.

Copy image Copy Code
SELECT SYSDATETIME() AS SYSDATETIME
    ,SYSDATETIMEOFFSET() AS SYSDATETIMEOFFSET
    ,SYSUTCDATETIME() AS SYSUTCDATETIME
    ,CURRENT_TIMESTAMP AS CURRENT_TIMESTAMP
    ,GETDATE() AS GETDATE
    ,GETUTCDATE() AS GETUTCDATE;

Here is the result set.

SYSDATETIME() 2007-04-30 13:10:02.0474381

SYSDATETIMEOFFSET()2007-04-30 13:10:02.0474381 -07:00

SYSUTCDATETIME() 2007-04-30 20:10:02.0474381

CURRENT_TIMESTAMP 2007-04-30 13:10:02.047

GETDATE() 2007-04-30 13:10:02.047

GETUTCDATE() 2007-04-30 20:10:02.047

B. Converting date and time to date

The following example shows you how to convert date and time values to date.

Copy image Copy Code
SELECT CONVERT (date, SYSDATETIME())
    ,CONVERT (date, SYSDATETIMEOFFSET())
    ,CONVERT (date, SYSUTCDATETIME())
    ,CONVERT (date, CURRENT_TIMESTAMP)
    ,CONVERT (date, GETDATE())
    ,CONVERT (date, GETUTCDATE());

Here is the result set.

2007-04-30

2007-04-30

2007-04-30

2007-04-30

2007-04-30

2007-04-30

C. Converting date and time to times

The following example shows you how to convert date and time values to time.

Copy image Copy Code
SELECT CONVERT (time, SYSDATETIME()) AS SYSDATETIME()
    ,CONVERT (time, SYSDATETIMEOFFSET()) AS SYSDATETIMEOFFSET()
    ,CONVERT (time, SYSUTCDATETIME()) AS SYSUTCDATETIME()
    ,CONVERT (time, CURRENT_TIMESTAMP) AS CURRENT_TIMESTAMP
    ,CONVERT (time, GETDATE()) AS GETDATE()
    ,CONVERT (time, GETUTCDATE()) AS GETUTCDATE();

Here is the result set.

SYSDATETIME() 13:18:45.3490361

SYSDATETIMEOFFSET()13:18:45.3490361

SYSUTCDATETIME() 20:18:45.3490361

CURRENT_TIMESTAMP 13:18:45.3470000

GETDATE() 13:18:45.3470000

GETUTCDATE() 20:18:45.3470000

See Also

Reference

Functions (SQL Server PDW)


SYSTEM_USER (SQL Server PDW)

[image]

SYSTEM_USER (SQL Server PDW)
Send Feedback

Allows a system-supplied value for the current login to be inserted into a table.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SYSTEM_USER

Return Types

nchar

Remarks

You can use the SYSTEM_USER function as any standard function.

If the user name and login name are different, SYSTEM_USER returns the login name.

SYSTEM_USER returns the name of the currently executing context.

Examples

The following example returns the current value of SYSTEM_USER.

Copy image Copy Code
SELECT SYSTEM_USER;


SYSUTCDATETIME (SQL Server PDW)

[image]

SYSUTCDATETIME (SQL Server PDW)
See Also Send Feedback

Returns a datetime2 value that contains the date and time of the computer on which the instance of SQL Server is running. The date and time is returned as UTC time (Coordinated Universal Time). The fractional second precision specification has a range from 1 to 7 digits. The default precision is 7 digits.

Note Note

SYSDATETIME and SYSUTCDATE have more fractional seconds precision than GETDATE and GETUTCDATE. SYSDATETIMEOFFSET includes the system time zone offset. SYSDATETIME, SYSUTCDATE, and SYSDATETIMEOFFSET can be assigned to a variable of any one of the date and time types.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SYSUTCDATETIME ( )

Return Type

datetime2

Remarks

Transact-SQL statements can refer to SYSUTCDATETIME anywhere they can refer to a datetime2 expression.

SYSUTCDATETIME is a nondeterministic function. Views and expressions that reference this function in a column cannot be indexed.

Examples

The following examples use the six SQL Server system functions that return current date and time to return the date, time, or both. The values are returned in series; therefore, their fractional seconds might be different.

A. Showing the formats that are returned by the date and time functions

The following example shows the different formats that are returned by the date and time functions.

Copy image Copy Code
SELECT SYSDATETIME() AS SYSDATETIME
    ,SYSDATETIMEOFFSET() AS SYSDATETIMEOFFSET
    ,SYSUTCDATETIME() AS SYSUTCDATETIME
    ,CURRENT_TIMESTAMP AS CURRENT_TIMESTAMP
    ,GETDATE() AS GETDATE
    ,GETUTCDATE() AS GETUTCDATE;

Here is the result set.

SYSDATETIME() 2007-04-30 13:10:02.0474381

SYSDATETIMEOFFSET()2007-04-30 13:10:02.0474381 -07:00

SYSUTCDATETIME() 2007-04-30 20:10:02.0474381

CURRENT_TIMESTAMP 2007-04-30 13:10:02.047

GETDATE() 2007-04-30 13:10:02.047

GETUTCDATE() 2007-04-30 20:10:02.047

B. Converting date and time to date

The following example shows you how to convert date and time values to date.

Copy image Copy Code
SELECT CONVERT (date, SYSDATETIME())
    ,CONVERT (date, SYSDATETIMEOFFSET())
    ,CONVERT (date, SYSUTCDATETIME())
    ,CONVERT (date, CURRENT_TIMESTAMP)
    ,CONVERT (date, GETDATE())
    ,CONVERT (date, GETUTCDATE());

Here is the result set.

2007-04-30

2007-04-30

2007-04-30

2007-04-30

2007-04-30

2007-04-30

C. Converting date and time values to time

The following example shows you how to convert date and time values to time.

DECLARE @DATETIME DATETIME = GetDate();

DECLARE @TIME TIME

SELECT @TIME = CONVERT(time, @DATETIME)

SELECT @TIME AS 'Time', @DATETIME AS 'Date Time'

Here is the result set.

Time Date Time

13:49:33.6330000 2009-04-22 13:49:33.633

See Also

Reference

Functions (SQL Server PDW)


TAN (SQL Server PDW)

[image]

TAN (SQL Server PDW)
See Also Send Feedback

Returns the tangent of the input expression SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

TAN ( float_expression )

Arguments

float_expression

An expression of type float or of a type that can be implicitly converted to float, interpreted as number of radians. float_expression must be a number between -1.0 and 1.0, inclusive.

Return Types

float

Examples

The following example returns the tangent of .45.

Copy image Copy Code
SELECT TAN(.45);

Here is the result set.

--------

0.48

See Also

Reference

Functions (SQL Server PDW)
ATAN (SQL Server PDW)
Expressions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


TERTIARY_WEIGHTS (SQL Server PDW)

[image]

TERTIARY_WEIGHTS (SQL Server PDW)
See Also Send Feedback

Returns a binary string of weights for each character in a non-Unicode string expression defined with an SQL tertiary collation.

Note Note

APS does not currently support SQL tertiary collations.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

TERTIARY_WEIGHTS( non_Unicode_character_string_expression )

Arguments

non_Unicode_character_string_expression

Is a string expression of type char, varchar, or varchar(max) defined on a tertiary SQL collation.

Return Types

TERTIARY_WEIGHTS returns varbinary when non_Unicode_character_string_expression is char or varchar, and returns varbinary(max) when non_Unicode_character_string_expression is varchar(max).

See Also

Reference

Functions (SQL Server PDW)

Concepts

Collations (SQL Server PDW)


TIMEFROMPARTS (SQL Server PDW)

[image]

TIMEFROMPARTS (SQL Server PDW)
Send Feedback

Returns a time value for the specified time and with the specified precision.

Topic link icon Transact-SQL Syntax Conventions

Applies to: SQL Server (SQL Server 2012 through current version), Azure SQL Database (Initial release through current release).

Syntax

TIMEFROMPARTS ( hour, minute, seconds, fractions, precision )

Arguments

hour

Integer expression specifying hours.

minute

Integer expression specifying minutes.

seconds

Integer expression specifying seconds.

fractions

Integer expression specifying fractions.

precision

Integer literal specifying the precision of the time value to be returned.

Return Types

time ( precision )

Remarks

TIMEROMPARTS returns a fully initialized time value. If the arguments are invalid, then an error is raised. If any of the parameters are null, null is returned. However, if the precision argument is null, then an error is raised.

The fractions argument depends on the precision argument. For example, if precision is 7, then each fraction represents 100 nanoseconds; if precision is 3, then each fraction represents a millisecond. If the value of precision is zero, then the value of fractions must also be zero; otherwise, an error is raised.

This function can be remoted to SQL Server 2012 servers and higher. It cannot be remoted to servers that have a version lower thanSQL Server 2012.

Examples

A. Simple example without fractions of a second

Copy image Copy Code
SELECT TIMEFROMPARTS ( 23, 59, 59, 0, 0 ) AS Result;

Here is the result set.

Copy image Copy Code
Result
--------------------
23:59:59.0000000

(1 row(s) affected)

B. Example with fractions of a second

The following example demonstrates the use of the fractions and precision parameters:

  1. When fractions has a value of 5 and precision has a value of 1, then the value of fractions represents 5/10 of a second.

  2. When fractions has a value of 50 and precision has a value of 2, then the value of fractions represents 50/100 of a second.

  3. When fractions has a value of 500 and precision has a value of 3, then the value of fractions represents 500/1000 of a second.

Transact-SQL Copy image Copy Code
SELECT TIMEFROMPARTS ( 14, 23, 44, 5, 1 );
SELECT TIMEFROMPARTS ( 14, 23, 44, 50, 2 );
SELECT TIMEFROMPARTS ( 14, 23, 44, 500, 3 );
GO

Here is the result set.

Copy image Copy Code
----------------
14:23:44.5

(1 row(s) affected)

----------------
14:23:44.50

(1 row(s) affected)

----------------
14:23:44.500

(1 row(s) affected)


TODATETIMEOFFSET (SQL Server PDW)

[image]

TODATETIMEOFFSET (SQL Server PDW)
See Also Send Feedback

Returns a datetimeoffset value that is translated from a datetime2 expression.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

TODATETIMEOFFSET ( expression , time_zone )

Arguments

expression

Is an expression that resolves to a datetime2 value.

Note Note

The expression cannot be of type text, ntext, or image because these types cannot be implicitly converted to varchar or nvarchar.

time_zone

Is an expression that represents the time zone offset in minutes (if an integer), for example -120, or hours and minutes (if a string), for example ‘+13.00’. The range is +14 to -14 (in hours). The expression is interpreted in local time for the specified time_zone.

Note Note

If expression is a character string, it must be in the format {+|-}TZH:THM.

Return Type

datetimeoffset. The fractional precision is the same as the datetime argument.

Examples

A. Changing the time zone offset of the current date and time

The following example changes the zone offset of the current date and time to time zone -07:00.

Copy image Copy Code
DECLARE @todaysDateTime datetime2;
SET @todaysDateTime = GETDATE();
SELECT TODATETIMEOFFSET (@todaysDateTime, '-07:00');
-- RETURNS 2007-08-30 15:51:34.7030000 -07:00

B. Changing the time zone offset in minutes

The following example changes the current time zone to -120 minutes.

Copy image Copy Code
DECLARE @todaysDate datetime2;
SET @todaysDate = GETDATE();
SELECT TODATETIMEOFFSET (@todaysDate, -120);
-- RETURNS 2007-08-30 15:52:37.8770000 -02:00

C. Adding a 13-hour time zone offset

The following example adds a 13-hour time zone offset to a date and time.

Copy image Copy Code
DECLARE @dateTime datetimeoffset(7)= '2007-08-28 18:00:30';
SELECT TODATETIMEOFFSET (@dateTime, '+13:00');
-- RETURNS 2007-08-28 18:00:30.0000000 +13:00

See Also

Reference

Functions (SQL Server PDW)


TYPE_ID (SQL Server PDW)

[image]

TYPE_ID (SQL Server PDW)
See Also Send Feedback

Returns the ID for a specified data type name.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

TYPE_ID ( [ schema_name ] type_name )

Arguments

type_name

The name of the data type. type_name is of type nvarchar.

Return Types

int

Exceptions

Returns NULL on error or if a caller does not have permission to view the object.

In SQL Server PDW, a user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as TYPE_ID may return NULL if the user does not have any permission on the object.

Remarks

TYPE_ID returns NULL if the type name is not valid, or if the caller does not have sufficient permission to reference the type.

Examples

The following example returns the TYPE ID for the datetime system data type.

Copy image Copy Code
SELECT TYPE_NAME(TYPE_ID('datetime')) AS typeName,
    TYPE_ID('datetime') AS typeID FROM table1;

See Also

Reference

TYPE_NAME (SQL Server PDW)
Functions (SQL Server PDW)


TYPE_NAME (SQL Server PDW)

[image]

TYPE_NAME (SQL Server PDW)
See Also Send Feedback

Returns the unqualified type name of a specified type ID.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

TYPE_NAME ( type_id )

Arguments

type_id

The ID of the type that will be used. type_id is an int, and it can refer to a type in any schema that the caller has permission to access.

Return Types

sysname

Exceptions

Returns NULL on error or if a caller does not have permission to view the object.

In SQL Server PDW, a user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as TYPE_NAME may return NULL if the user does not have any permission on the object.

Remarks

TYPE_NAME will return NULL when type_id is not valid or when the caller does not have sufficient permission to reference the type.

TYPE_NAME works for system data types and also for user-defined data types. The type can be contained in any schema, but an unqualified type name is always returned. This means the name does not have the schema. prefix.

System functions can be used in the select list, in the WHERE clause, and anywhere an expression is allowed.

Examples

The following example returns the TYPE ID for the data type with id 1.

Copy image Copy Code
SELECT TYPE_NAME(36) AS Type36, TYPE_NAME(239) AS Type239;
GO

For a list of types, query sys.types.

Copy image Copy Code
SELECT * FROM sys.types;
GO

See Also

Reference

Functions (SQL Server PDW)


TYPEPROPERTY (SQL Server PDW)

[image]

TYPEPROPERTY (SQL Server PDW)
See Also Send Feedback

Returns information about a data type.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

TYPEPROPERTY (type , property)

Arguments

type

The name of the data type.

property

The type of information to be returned for the data type. property can be one of the following values.

Property

Description

Value returned

AllowsNull

Data type allows for null values.

1 = True

0 = False

NULL = Data type not found.

OwnerId

Owner of the type.

Note Note

The schema owner is not necessarily the type owner.

Nonnull = The database user ID of the type owner.

NULL = Unsupported type, or type ID is not valid.

Precision

Precision for the data type.

The number of digits or characters.

-1 = xml or large value data type

NULL = Data type not found.

Scale

Scale for the data type.

The number of decimal places for the data type.

NULL = Data type is not numeric or not found.

UsesAnsiTrim

ANSI padding setting was ON when the data type was created.

1 = True

0 = False

NULL = Data type not found, or it is not a binary or string data type.

Return Types

int

Exceptions

Returns NULL on error or if a caller does not have permission to view the object.

In SQL Server PDW, a user can only view the metadata of securables that the user owns or on which the user has been granted permission. This means that metadata-emitting, built-in functions such as TYPEPROPERTY may return NULL if the user does not have any permission on the object.

Examples

The following example returns the precision or number of digits for the tinyint data type.

Copy image Copy Code
SELECT TYPEPROPERTY( 'tinyint', 'PRECISION');

See Also

Reference

Functions (SQL Server PDW)


UNICODE (SQL Server PDW)

[image]

UNICODE (SQL Server PDW)
See Also Send Feedback

Returns the integer value, as defined by the Unicode standard, for the first character of the input expression.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

UNICODE ( 'ncharacter_expression' )

Arguments

'ncharacter_expression'

Is an nchar or nvarchar expression.

Return Types

int

Remarks

When using SC collations, UNICODE returns a UTF-16 codepoint in the range 0 through 0x10FFFF.

Examples

A. Using UNICODE and the NCHAR function

The following example uses the UNICODE and NCHAR functions to print the UNICODE value of the first character of the Åkergatan 24-character string, and to print the actual first character, Å.

Copy image Copy Code
DECLARE @nstring nchar(12);
SET @nstring = N'Åkergatan 24';
SELECT UNICODE(@nstring), NCHAR(UNICODE(@nstring));

Here is the result set.

Copy image Copy Code
----------- -
197         Å

B. Using SUBSTRING, UNICODE, and CONVERT

The following example uses the SUBSTRING, UNICODE, and CONVERT functions to print the character number, the Unicode character, and the UNICODE value of each of the characters in the string Åkergatan 24.

Copy image Copy Code
-- The @position variable holds the position of the character currently
-- being processed. The @nstring variable is the Unicode character
-- string to process.
DECLARE @position int, @nstring nchar(12);
-- Initialize the current position variable to the first character in
-- the string.
SET @position = 1;
-- Initialize the character string variable to the string to process.
-- Notice that there is an N before the start of the string, which
-- indicates that the data following the N is Unicode data.
SET @nstring = N'Åkergatan 24';
-- Print the character number of the position of the string you are at,
-- the actual Unicode character you are processing, and the UNICODE
-- value for this particular character.
PRINT 'Character #' + ' ' + 'Unicode Character' + ' ' + 'UNICODE Value';
WHILE @position <= DATALENGTH(@nstring)
-- While these are still characters in the character string,
   BEGIN;
   SELECT @position,
      CONVERT(char(17), SUBSTRING(@nstring, @position, 1)),
      UNICODE(SUBSTRING(@nstring, @position, 1));
   SELECT @position = @position + 1;
   END;

Here is the result set.

Copy image Copy Code
Character # Unicode Character UNICODE Value

----------- ----------------- -----------
1           Å                 197

----------- ----------------- -----------
2           k                 107

----------- ----------------- -----------
3           e                 101

----------- ----------------- -----------
4           r                 114

----------- ----------------- -----------
5           g                 103

----------- ----------------- -----------
6           a                 97

----------- ----------------- -----------
7           t                 116

----------- ----------------- -----------
8           a                 97

----------- ----------------- -----------
9           n                 110

----------- ----------------- -----------
10                            32

----------- ----------------- -----------
11          2                 50

----------- ----------------- -----------
12          4                 52

See Also

Reference

Functions (SQL Server PDW)

Concepts

Collations (SQL Server PDW)


UPPER (SQL Server PDW)

[image]

UPPER (SQL Server PDW)
See Also Send Feedback

Returns a character expression with lowercase character data converted to uppercase in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

UPPER ( character_expression )

Arguments

character_expression

An expression of character data. character_expression can be a constant or column of character data.

character_expression must be of a data type that is implicitly convertible to varchar or nvarchar. Otherwise, use CAST to explicitly convert character_expression.

Return Types

varchar or nvarchar

Examples

The following example uses the UPPER and RTRIM functions to return the last name of people in the dbo.DimEmployee table so that it is in uppercase, trimmed, and concatenated with the first name.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT UPPER(RTRIM(LastName)) + ', ' + FirstName AS Name
FROM dbo.DimEmployee
ORDER BY LastName;

Here is a partial result set.

Name

------------------------------

ABBAS, Syed

ABERCROMBIE, Kim

ABOLROUS, Hazem

See Also

Reference

Expressions (SQL Server PDW)
CAST and CONVERT (SQL Server PDW)
RTRIM (SQL Server PDW)
LOWER (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


USER (SQL Server PDW)

[image]

USER (SQL Server PDW)
See Also Send Feedback

Allows a system-supplied value for the database user name of the current user to be inserted into a table when no default value is specified.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

USER

Return Types

char

Remarks

USER provides the same functionality as the USER_NAME system function.

Use USER with DEFAULT constraints in either the CREATE TABLE or ALTER TABLE statements, or use as any standard function.

USER always returns the name of the current context. When called after an EXECUTE AS statement, USER returns the name of the impersonated context.

If a Windows principal accesses the database by way of membership in a group, USER returns the name of the Windows principal instead of the name of the group.

Examples

A. Using USER to return the database user name

The following example declares a variable as char, assigns the current value of USER to it, and then prints the variable with a text description.

Copy image Copy Code
DECLARE @usr char(30)
SET @usr = user
SELECT 'The current user''s database username is: '+ @usr
GO

Here is the result set.

-----------------------------------------------------------------------

The current user's database username is: dbo

(1 row(s) affected)

See Also

Reference

USER_NAME (SQL Server PDW)
SUSER_SNAME (SQL Server PDW)
HAS_DBACCESS (SQL Server PDW)
sys.database_principals (SQL Server PDW)
sys.database_permissions (SQL Server PDW)


USER_NAME (SQL Server PDW)

[image]

USER_NAME (SQL Server PDW)
See Also Send Feedback

Returns a database user name from a specified identification number.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

USER_NAME ( [ id ] )

Arguments

id

The identification number associated with a database user. idis int. The parentheses are required. When id is omitted, the current user in the current context is assumed.

Return Types

nvarchar(128)

Remarks

If the parameter contains the word NULL will return NULL.

Examples

A. Using USER_NAME

The following example returns the user name for user ID 13.

Copy image Copy Code
SELECT USER_NAME(13);

B. Using USER_NAME without an ID

The following example finds the name of the current user without specifying an ID.

Copy image Copy Code
SELECT USER_NAME();

Here is the result set for a currently logged-in user.

Here is the result set.

Copy image Copy Code
------------------------------
User7

C. Using USER_NAME in the WHERE clause

The following example finds the row in sysusers in which the name is equal to the result of applying the system function USER_NAME to user identification number 1.

Copy image Copy Code
SELECT name FROM sysusers WHERE name = USER_NAME(1);

Here is the result set.

Copy image Copy Code
name
------------------------------
User7

See Also

Reference

USER (SQL Server PDW)
SUSER_SNAME (SQL Server PDW)
HAS_DBACCESS (SQL Server PDW)
sys.database_principals (SQL Server PDW)
sys.database_permissions (SQL Server PDW)


VAR (SQL Server PDW)

[image]

VAR (SQL Server PDW)
See Also Send Feedback

Returns the statistical variance of all values within the specified expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Aggregate Function Syntax
VAR ( [ ALL | DISTINCT ] expression )
-- Analytic Function Syntax
VAR (expression) OVER ( [ partition_by_clause ] order_by_clause  )

Arguments

ALL

Applies the aggregate function to all values. ALL is the default.

DISTINCT

Specifies that each unique value is considered.

expression

An expression of the exact numeric or approximate numeric data type category, except for the bit data type. Aggregate functions, analytic functions, and subqueries are not permitted.

partition_by_clause

Divides the result set produced by the FROM (SQL Server PDW) clause into partitions to which the VAR function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

order_by_clause

Determines the order in which the VAR values are applied to the rows in a partition. An integer cannot represent a column in order_by_clause. When the order_by_clause is absent, all rows in the same partition are assigned the same value for the VAR() OVER computation. When the order_by_clause is present, the VAR() OVER computation gives you a “cumulative var” in the partition. Within a partition, rows with identical values for all elements of the order_by_clause will return identical results for the VAR() OVER computation. For more information, see ORDER BY (SQL Server PDW).

Return Types

float

General Remarks

If VAR is used on all items in a SELECT statement, each value in the result set is included in the calculation. VAR can be used with numeric columns only. Null values are ignored.

Limitations and Restrictions

The VAR() OVER analytic function does not support the use of the ALL or DISTINCT modifiers.

Examples

Using VAR

The following example returns the statistical variance of the sales quota values in the table dbo.FactSalesQuota. The first column contains the variance of all distinct values and the second column contains the variance of all values including any duplicates values.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT VAR(DISTINCT SalesAmountQuota)AS Distinct_Values, VAR(SalesAmountQuota) AS All_Values
FROM dbo.FactSalesQuota;

Here is the result set.

Distinct_Values All_Values

---------------- ----------------

159180469909.18 158762853821.10

B. Using VAR with OVER

The following example returns the statistical variance of the sales quota values for each quarter in a calendar year. Notice that the ORDER BY in the OVER clause orders the statistical variance and the ORDER BY of the SELECT statement orders the result set.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT CalendarYear AS Year, CalendarQuarter AS Quarter, SalesAmountQuota AS SalesQuota,
       VAR(SalesAmountQuota) OVER (ORDER BY CalendarYear, CalendarQuarter) AS Variance
FROM dbo.FactSalesQuota
WHERE EmployeeKey = 272 AND CalendarYear = 2002
ORDER BY CalendarQuarter;

Here is the result set.

Year Quarter SalesQuota Variance

---- ------- ---------------------- -------------------

2002 1 91000.0000 null

2002 2 140000.0000 1200500000.00

2002 3 70000.0000 1290333333.33

2002 4 154000.0000 1580250000.00

See Also

Reference

Functions (SQL Server PDW)


VARP (SQL Server PDW)

[image]

VARP (SQL Server PDW)
See Also Send Feedback

Returns the statistical variance for the population of all values within the specified expression in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Aggregate Function Syntax
VARP ( [ ALL | DISTINCT ] expression )
-- Analytic Function Syntax
VARP (expression) OVER ( [ partition_by_clause ] order_by_clause  )

Arguments

ALL

Applies the aggregate function to all values. ALL is the default.

DISTINCT

Specifies that each unique value is considered.

expression

An expression of the exact numeric or approximate numeric data type category, except for the bit data type. Aggregate functions, analytic functions, and subqueries are not permitted.

partition_by_clause

Divides the result set produced by the FROM (SQL Server PDW) clause into partitions to which the VARP function is applied. For the PARTITION BY syntax, see OVER Clause (SQL Server PDW).

order_by_clause

Determines the order in which the VARP values are applied to the rows in a partition. An integer cannot represent a column in order_by_clause. When the order_by_clause is absent, all rows in the same partition are assigned the same value for the VARP() OVER computation. When the order_by_clause is present, the VARP() OVER computation gives you a “cumulative varp” in the partition. Within a partition, rows with identical values for all elements of the order_by_clause will return identical results for the VARP() OVER computation. For more information, see ORDER BY (SQL Server PDW).

Return Types

float

General Remarks

If VARP is used on all items in a SELECT statement, each value in the result set is included in the calculation. VARP can be used with numeric columns only. Null values are ignored.

Limitations and Restrictions

The VARP() OVER analytic function does not support the use of the ALL or DISTINCT modifiers.

Examples

Using VARP

The following example returns the VARPof the sales quota values in the table dbo.FactSalesQuota. The first column contains the variance of all distinct values and the second column contains the variance of all values including any duplicates values.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT VARP(DISTINCT SalesAmountQuota)AS Distinct_Values, VARP(SalesAmountQuota) AS All_Values
FROM dbo.FactSalesQuota;

Here is the result set.

Distinct_Values All_Values

---------------- ----------------

158146830494.18 157788848582.94

B. Using VARP with OVER

The following example returns the VARP of the sales quota values for each quarter in a calendar year. Notice that the ORDER BY in the OVER clause orders the statistical variance and the ORDER BY of the SELECT statement orders the result set.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT CalendarYear AS Year, CalendarQuarter AS Quarter, SalesAmountQuota AS SalesQuota,
       VARP(SalesAmountQuota) OVER (ORDER BY CalendarYear, CalendarQuarter) AS Variance
FROM dbo.FactSalesQuota
WHERE EmployeeKey = 272 AND CalendarYear = 2002
ORDER BY CalendarQuarter;

Here is the result set.

Year Quarter SalesQuota Variance

---- ------- ---------------------- -------------------

2002 1 91000.0000 0.00

2002 2 140000.0000 600250000.00

2002 3 70000.0000 860222222.22

2002 4 154000.0000 1185187500.00

See Also

Reference

Functions (SQL Server PDW)


VERSION (SQLServer PDW)

[image]

VERSION (SQLServer PDW)
See Also Send Feedback

Returns the version of SQL Server PDW running on the appliance.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

VERSION ( )

General Remarks

A table name must be specified in a FROM clause for this function to return results. A result row will be returned for each row in the result set for the query; use TOP to limit the number of returned rows.

Examples

The following example returns the version number.

Copy image Copy Code
SELECT VERSION();

See Also

Reference

SESSION_ID (SQL Server PDW)
DB_NAME (SQL Server PDW)


YEAR (SQL Server PDW)

[image]

YEAR (SQL Server PDW)
See Also Send Feedback

Returns an integer that represents the year of the specified date in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

YEAR ( date )

Arguments

date

An expression that can be resolved to a time, date, smalldatetime, datetime, datetime2, or datetimeoffset value. The date argument can be an expression, column expression, or string literal.

Return Types

int

Return Value

YEAR returns the same value as DATEPART (year, date).

If date only contains a time part, the return value is 1900, the base year.

Examples

The following statement returns 2010. This is the number of the year.

Copy image Copy Code
SELECT YEAR('2010-07-20T01:01:01.1234');

The following statement returns 1900, 1, 1. The argument for date is the number 0. SQL Server interprets 0 as January 1, 1900.

Copy image Copy Code
SELECT TOP 1 YEAR(0), MONTH(0), DAY(0);

See Also

Reference

MONTH (SQL Server PDW)
DAY (SQL Server PDW)


XACT_STATE (SQL Server PDW)

[image]

XACT_STATE (SQL Server PDW)
See Also Send Feedback

Is a scalar function that reports the user transaction state of a current running request. XACT_STATE indicates whether the request has an active user transaction, and whether the transaction is capable of being committed.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

XACT_STATE()

Return Type

smallint

Remarks

XACT_STATE returns the following values.

Return value

Meaning

1

The current request has an active user transaction. The request can perform any actions, including writing data and committing the transaction.

0

There is no active user transaction for the current request.

-1

This state exists in SQL Server, but is not exposed in PDW. The definition below is for comparison to SQL Server behavior.

The current request has an active user transaction, but an error has occurred that has caused the transaction to be classified as an uncommittable transaction. The request cannot commit the transaction or roll back to a savepoint; it can only request a full rollback of the transaction. The request cannot perform any write operations until it rolls back the transaction. The request can only perform read operations until it rolls back the transaction. After the transaction has been rolled back, the request can perform both read and write operations and can begin a new transaction.

When a batch finishes running, the Database Engine will automatically roll back any active uncommittable transactions. If no error message was sent when the transaction entered an uncommittable state, when the batch finishes, an error message will be sent to the client application. This message indicates that an uncommittable transaction was detected and rolled back.

-2

The current request has an active user transaction, but an error has occurred that caused the transaction to be classified as a rollback-only transaction.

When transaction reaches this state, you can execute the following statements: SET, DECLARE, PRINT, RAISERROR, THROW, and control-flow statements (IF/ELSE, WHILE, TRY…CATCH).

Transactions in this state are automatically rolled-back at the end of the batch, unless an explicit ROLLBACK is requested earlier.

Both the XACT_STATE and @@TRANCOUNT functions can be used to detect whether the current request has an active user transaction. @@TRANCOUNT cannot be used to determine whether that transaction has been classified as an uncommittable transaction. XACT_STATE cannot be used to determine whether there are nested transactions.

Example of an error handling flow and progression of XACT_STATE:

Step | Statement | XACT_STATE | Comments
1 | BEGIN TRAN | 0 | Beginning transaction
2 | BEGIN TRY | 1 | Transaction is active.
3 | SELECT 1/0 | 1 | Division by zero happens here.
 | ... | N/A | Control passed to CATCH block.
 | END TRY | N/A |
4 | BEGIN CATCH | -2 | Transaction is now in a rollback-only state
5 | DECLARE @err int = @@ERROR | -2 | DECLARE, SET, control flow statements are allowed.
6 | DECLARE @xact int = XACT_STATE() | -2 | We cache error and transaction states for later use.
6 | END CATCH | -2 |
7 | -- SELECT @err, @xact | -2 | SELECTs are still prohibited here.
8 | ROLLBACK | -2 | Rolling back the transaction.
9 | SELECT @err, @xact | 0 | No active transactions, SELECTs are allowed.

See Also

Reference

TRY...CATCH (SQL Server PDW)
THROW (SQL Server PDW)


Batches, Control-of-Flow, and Variables (SQL Server PDW)

[image]

Batches, Control-of-Flow, and Variables (SQL Server PDW)
See Also Send Feedback

A batch is a group of one or more SQL statements sent as part of a single request from an application to SQL Server PDW for execution. Batches allow the use of conditional statements and variables in SQL Server PDW.

Contents

  • General Remarks

  • Security

  • Examples

General Remarks

The statements in a SQL Server PDW batch are compiled and run sequentially. Each statement in a batch goes through the normal execution path, and through the normal queues and concurrency locks. This means that other concurrent users can preempt the execution of the batch in between the batch statements.

Transactions are supported within batches. This means that BEGIN TRANSACTION, SET AUTOCOMMIT, COMMIT and ROLLBACK can be used within a batch.

Each SQL statement within a batch should be terminated with a semicolon. This requirement is not enforced, but the ability to end a statement without a semicolon is deprecated and may be removed in a future version of SQL Server PDW.

A compilation or run-time error, such as a syntax error or arithmetic overflow error, prevents the batch from proceeding.

A batch abort due to a runtime error during a transaction does not automatically rollback the transaction unless the specific statement level error that caused the batch to abort also aborts the transaction.

Each batch has no ability to reference any variables declared in another batch.

If comments are used, they must start and end within the same batch.

CREATE VIEW statements cannot be combined with other statements in a batch.

Error Handling

Statements useful for error handling include: PRINT, RAISERROR, THROW, and TRY...CATCH.

Security

Batch files might contain credentials stored in plain text. Credentials may be echoed to the screen of the user during batch execution.

Examples

A. Batches within Transactions

The following example shows several batches combined into one transaction. The BEGIN TRANSACTION and COMMIT statements delimit the transaction boundaries. The BEGIN TRANSACTION, USE, SELECT, and COMMIT statements are all in their own single-statement batches. All of the INSERT statements are included in one batch.

This example INSERTs values into an existing mycompanies table in the mydatabase database, then SELECTs the values back for display to the user. The CREATE DATABASE and CREATE TABLE commands to prepare these objects are also shown below.

Copy image Copy Code
CREATE DATABASE mydatabase
   WITH
   (REPLICATED_SIZE = 1.5 GB,
   DISTRIBUTED_SIZE = 5.25 GB,
   LOG_SIZE = 10);

CREATE TABLE mydatabase..mycompanies
(
 id_num int,
 company_name nvarchar(100)
);

BEGIN TRANSACTION
    BEGIN
        USE mydatabase;
    END
    BEGIN
        INSERT INTO mycompanies
           VALUES (1, N'A Bike Store');
        INSERT INTO mycompanies
           VALUES (2, N'Progressive Sports');
        INSERT INTO mycompanies
           VALUES (3, N'Modular Cycle Systems');
        INSERT INTO mycompanies
           VALUES (4, N'Advanced Bike Components');
        INSERT INTO mycompanies
           VALUES (5, N'Metropolitan Sports Supply');
        INSERT INTO mycompanies
           VALUES (6, N'Aerobic Exercise Company');
        INSERT INTO mycompanies
           VALUES (7, N'Associated Bikes');
        INSERT INTO mycompanies
           VALUES (8, N'Exemplary Cycles');
    END
    BEGIN
        SELECT id_num, company_name
        FROM mycompanies
        ORDER BY company_name ASC;
    END
COMMIT;

B. Specifying Batches through Database APIs

  • In ADO, a batch request can be specified by setting the CommandText property of a Command object to a string of multiple statements:

    Copy image Copy Code
    Dim Cmd As New ADODB.Command
Set Cmd.ActiveConnection = Cn
Cmd.CommandText = "SELECT * FROM dbo.Vendor; SELECT * FROM dbo.Product"
Cmd.CommandType = adCmdText
Cmd.Execute

  • In OLE DB, a batch can be sent as a string of SQL statements enclosed in the string used to set the command text:

    Copy image Copy Code
    WCHAR* wszSQLString =
L"SELECT * FROM dbo.Employee; SELECT * FROM dbo.Product";
hr = pICommandText->SetCommandText
      (DBGUID_DBSQL, wszSQLString)

  • In ODBC, a batch can be sent as a string of Transact-SQL statements enclosed on a SQLPrepare or SQLExecDirect call:

    Copy image Copy Code
    SQLExecDirect(hstmt1,
   "SELECT * FROM dbo.Employee; SELECT * FROM dbo.Product",
SQL_NTS):

See Also


BEGIN...END (SQL Server PDW)

[image]

BEGIN...END (SQL Server PDW)
See Also Send Feedback

Encloses a series of SQL statements in SQL Server PDW. BEGIN and END are provided for compatibility with SQL Server. BEGIN and END are control-of-flow language keywords.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

BEGIN
     {
    sql_statement | statement_block
     }
END

Arguments

{ sql_statement| statement_block }

Is any valid SQL statement or statement grouping as defined by using a statement block.

General Remarks

BEGIN...END blocks can be nested.

Although all SQL statements are valid within a BEGIN...END block, certain SQL statements should not be grouped together within the same batch, or statement block. For more information, see Batches, Control-of-Flow, and Variables (SQL Server PDW) and the individual statements used.

Examples

In the following example, BEGIN and END define a series of SQL statements that run together. If the BEGIN...END block are not included, the following example will be in a continuous loop.

Copy image Copy Code
USE AdventureWorksDW2008;

DECLARE @Iteration Integer = 0
WHILE @Iteration <10
BEGIN
    SELECT FirstName, MiddleName
    FROM dbo.DimCustomer WHERE LastName = 'Adams';
SET @Iteration += 1
END;

See Also


BREAK (SQL Server PDW)

[image]

BREAK (SQL Server PDW)
See Also Send Feedback

Exits the innermost loop in a WHILE statement or an IF…ELSE statement inside a WHILE loop. Any statements appearing after the END keyword, marking the end of the loop, are run.

Examples

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

WHILE ((SELECT AVG(ListPrice) FROM dbo.DimProduct) < $300)
BEGIN
    UPDATE DimProduct
        SET ListPrice = ListPrice * 2;
     IF ((SELECT MAX(ListPrice) FROM dbo.DimProduct) > $500)
         BREAK;
END

See Also


DECLARE (SQL Server PDW)

[image]

DECLARE (SQL Server PDW)
See Also Send Feedback

Variables are declared in the body of a batch with the DECLARE statement and are assigned values by using the SET statement or the SELECT statement. After declaration, all variables are initialized as NULL, unless a value is provided as part of the declaration.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DECLARE
{{ @local_variable [AS] data_type } [ = value [ COLLATE <collation_name> ] ] } [,...n]

Arguments

@local_variable

Is the name of a variable. Variable names must begin with an at (@) sign. Local variable names must comply with the rules for identifiers. For more information, see Object Naming Rules (SQL Server PDW).

data_type

Is any system-supplied data type. For a list of data types, see CREATE TABLE (SQL Server PDW). The sysname and sql_variant data types can be used for variables but cannot be used in user-defined tables.

=value

Assigns a value to the variable in-line. The value can be a constant or an expression, but it must either match the variable declaration type or be implicitly convertible to that type. For more information, see Expressions (SQL Server PDW).

General Remarks

Variables are often used in a batch as counters for WHILE or IF...ELSE blocks.

The scope of a local variable is the batch in which it is declared.

Caution note Caution

Comparing a null valued variable to a literal does not fail as does in SQL Server 2012.

Limitations and Restrictions

Variables can be used only in expressions, not in place of object names or keywords.

Using a variable in its own DECLARE statement is not supported and may result in inconsistent behavior. For example, the following example declares @p1 and then attempts to use @p1 later in the declaration statement. This fails, returning NULL.

Copy image Copy Code
DECLARE @p1 int = 100, @p2 int = (SELECT MAX (@p1) FROM sys.types);
SELECT MAX (@p1), MAX(@p2) FROM sys.types;

In SQL Server PDW, local variables of sql_variant type always have the PDW appliance-level collation, which is case-insensitive. This overrides an assigned collation when the variables are declared. Both of the following examples return 1. This behavior is different than SQL Server.

Copy image Copy Code
--@var1 and @var2 actually have the PDW-wide
--collation Latin1_General_100_CI_AS_KS_WS
DECLARE @var1 sql_variant = 'a' COLLATE Latin1_General_CS_AS_KS_WS;
DECLARE @var2 sql_variant = 'A' COLLATE Latin1_General_CS_AS_KS_WS
IF @var1 = @var2
    SELECT 1;
ELSE
    SELECT 2

DECLARE @var1 sql_variant = 'a' COLLATE Corsican_100_CI_AI;
DECLARE @var2 sql_variant = 'A' COLLATE Latin1_General_CS_AS_KS_WS
IF @var1 = @var2
    SELECT 1;
ELSE
    SELECT 2

Examples

A. Using DECLARE

The following example uses a local variable named @find to retrieve contact information for all last names beginning with Walt.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

DECLARE @find varchar(30);
/* Also allowed:
DECLARE @find varchar(30) = 'Man%';
*/
SET @find = 'Walt%';

SELECT LastName, FirstName, Phone
FROM DimEmployee
WHERE LastName LIKE @find;

B. Using DECLARE with two variables

The following example retrieves uses variables to specify the first and last names of employees in the DimEmployee table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

DECLARE @lastName varchar(30), @firstName varchar(30);

SET @lastName = 'Walt%';
SET @firstName = 'Bryan';

SELECT LastName, FirstName, Phone
FROM DimEmployee
WHERE LastName LIKE @lastName AND FirstName LIKE @firstName;

See Also


IF...ELSE (SQL Server PDW)

[image]

IF...ELSE (SQL Server PDW)
See Also Send Feedback

Imposes conditions on the execution of a SQL Server PDW statement or group of statements.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

IF Boolean_expression
     { sql_statement | statement_block }
[ ELSE
     { sql_statement | statement_block } ]

Arguments

Boolean_expression

Is an expression that returns TRUE or FALSE. If the Boolean expression contains a SELECT statement, the SELECT statement must be enclosed in parentheses.

{ sql_statement| statement_block }

Is any SQL statement or statement grouping as defined by using a statement block. Unless a statement block is used, the IF or ELSE condition can affect the performance of only one SQL statement.

To define a statement block, use the control-of-flow keywords BEGIN and END.

Return Values

Returns TRUE if the condition that follows an IF keyword evaluates to TRUE.

Returns FALSE if the condition that follows an IF keyword evaluates to FALSE.

General Remarks

The IF…ELSE construct can be used in batches and in ad hoc queries.

IF tests can be nested after another IF or following an ELSE. The limit to the number of nested levels depends on available memory.

Examples

The following example uses IF…ELSE to determine which of two responses to show the user, based on the weight of an item in the DimProduct table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

DECLARE @maxWeight float, @productKey integer
SET @maxWeight = 100.00
SET @productKey = 424
IF @maxWeight <= (SELECT Weight from DimProduct WHERE ProductKey=@productKey)
    (SELECT @productKey, EnglishDescription, Weight, 'This product is too heavy to ship and is only available for pickup.' FROM DimProduct WHERE ProductKey=@productKey)
ELSE
    (SELECT @productKey, EnglishDescription, Weight, 'This product is available for shipping or pickup.' FROM DimProduct WHERE ProductKey=@productKey)

See Also


RAISERROR (SQL Server PDW)

[image]

RAISERROR (SQL Server PDW)
See Also Send Feedback

Generates an error message and initiates error processing for the session. The error number will always be 50000. The message text is built dynamically. The message is returned as a server error message to the calling application or to an associated CATCH block of a TRY…CATCH construct. New applications should use THROW instead.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

RAISERROR ( { msg_str | @local_variable }
    { ,severity ,state }
    [ ,argument [ ,...n ] ] )
    [ WITH option [ ,...n ] ]

Arguments

msg_str

Is a user-defined message with formatting similar to the printf function in the C standard library. The error message can have a maximum of 2,047 characters. If the message contains 2,048 or more characters, only the first 2,044 are displayed and an ellipsis is added to indicate that the message has been truncated. Note that substitution parameters consume more characters than the output shows because of internal storage behavior. For example, the substitution parameter of %d with an assigned value of 2 actually produces one character in the message string but also internally takes up three additional characters of storage. This storage requirement decreases the number of available characters for message output.

RAISERROR raises an error message with an error number of 50000.

msg_str is a string of characters with optional embedded conversion specifications. Each conversion specification defines how a value in the argument list is formatted and placed into a field at the location of the conversion specification in msg_str. Conversion specifications have this format:

% [[flag] [width] [. precision] [{h | l}]] type

The parameters that can be used in msg_str are:

flag

Is a code that determines the spacing and justification of the substituted value.

Code

Prefix or justification

Description

- (minus)

Left-justified

Left-justify the argument value within the given field width.

+ (plus)

Sign prefix

Preface the argument value with a plus (+) or minus (-) if the value is of a signed type.

0 (zero)

Zero padding

Preface the output with zeros until the minimum width is reached. When 0 and the minus sign (-) appear, 0 is ignored.

# (number)

0x prefix for hexadecimal type of x or X

When used with the o, x, or X format, the number sign (#) flag prefaces any nonzero value with 0, 0x, or 0X, respectively. When d, i, or u are prefaced by the number sign (#) flag, the flag is ignored.

' ' (blank)

Space padding

Preface the output value with blank spaces if the value is signed and positive. This is ignored when included with the plus sign (+) flag.

width

Is an integer that defines the minimum width for the field into which the argument value is placed. If the length of the argument value is equal to or longer than width, the value is printed with no padding. If the value is shorter than width, the value is padded to the length specified in width.

An asterisk (*) means that the width is specified by the associated argument in the argument list, which must be an integer value.

precision

Is the maximum number of characters taken from the argument value for string values. For example, if a string has five characters and precision is 3, only the first three characters of the string value are used.

For integer values, precision is the minimum number of digits printed.

An asterisk (*) means that the precision is specified by the associated argument in the argument list, which must be an integer value.

{h | l} type

Is used with character types d, i, o, s, x, X, or u, and creates shortint (h) or longint (l) values.

Type specification

Represents

d or i

Signed integer

o

Unsigned octal

s

String

u

Unsigned integer

x or X

Unsigned hexadecimal

Note Note

These type specifications are based on the ones originally defined for the printf function in the C standard library. The type specifications used in RAISERROR message strings map to Transact-SQL data types, while the specifications used in printf map to C language data types. Type specifications used in printf are not supported by RAISERROR when Transact-SQL does not have a data type similar to the associated C data type. For example, the %p specification for pointers is not supported in RAISERROR because Transact-SQL does not have a pointer data type.

Note Note

To convert a value to the Transact-SQL bigint data type, specify %I64d.

@local_variable

Is a variable of any valid character data type that contains a string formatted in the same manner as msg_str. @local_variable must be char or varchar, or be able to be implicitly converted to these data types.

severity

Is the user-defined severity level associated with this message.

Severity levels from 0 through 18 can be specified by any user. Severity levels less than 0 are interpreted as 0.

state

Is an integer from 0 through 255. Negative values produce the state 0. Values larger than 255 generate an error.

argument

Are the parameters used in the substitution for variables defined in msg_str. There can be 0 or more substitution parameters, but the total number of substitution parameters cannot exceed 20. Each substitution parameter can be a local variable or any of these data types: tinyint, smallint, int, char, varchar, nchar, nvarchar, binary, or varbinary. No other data types are supported.

option

Is a custom option for the error. Currently NOWAIT is the only option

Value

Description

NOWAIT

Sends messages immediately to the client.

Remarks

The errors generated by RAISERROR operate the same as errors generated by the PDW engine. The values specified by RAISERROR are reported by the ERROR_MESSAGE, ERROR_NUMBER, ERROR_PROCEDURE, ERROR_SEVERITY, ERROR_STATE, and @@ERROR system functions. When RAISERROR is run with a severity of 11 or higher in a TRY block, it transfers control to the associated CATCH block. The error is returned to the caller if RAISERROR is run outside the scope of any TRY block.

CATCH blocks can use RAISERROR to rethrow the error that invoked the CATCH block by using system functions such as ERROR_NUMBER and ERROR_MESSAGE to retrieve the original error information. @@ERROR is set to 0 by default for messages with a severity from 1 through 10.

RAISERROR can be used as an alternative to PRINT to return messages to calling applications. RAISERROR supports character substitution similar to the functionality of the printf function in the C standard library, while the Transact-SQL PRINT statement does not. The PRINT statement is not affected by TRY blocks, while a RAISERROR run with a severity of 11 to 19 in a TRY block transfers control to the associated CATCH block. Specify a severity of 10 or lower to use RAISERROR to return a message from a TRY block without invoking the CATCH block.

Typically, successive arguments replace successive conversion specifications; the first argument replaces the first conversion specification, the second argument replaces the second conversion specification, and so on. For example, in the following RAISERROR statement, the first argument of N'number' replaces the first conversion specification of %s; and the second argument of 5 replaces the second conversion specification of %d.

Copy image Copy Code
RAISERROR (N'This is message %s %d.', -- Message text.
           10, -- Severity,
           1, -- State,
           N'number', -- First argument.
           5); -- Second argument.
-- The message text returned is: This is message number 5.
GO

If an asterisk (*) is specified for either the width or precision of a conversion specification, the value to be used for the width or precision is specified as an integer argument value. In this case, one conversion specification can use up to three arguments, one each for the width, precision, and substitution value.

For example, both of the following RAISERROR statements return the same string. One specifies the width and precision values in the argument list; the other specifies them in the conversion specification.

Copy image Copy Code
RAISERROR (N'<<%*.*s>>', -- Message text.
           10, -- Severity,
           1, -- State,
           7, -- First argument used for width.
           3, -- Second argument used for precision.
           N'abcde'); -- Third argument supplies the string.
-- The message text returned is: <<    abc>>.
GO
RAISERROR (N'<<%7.3s>>', -- Message text.
           10, -- Severity,
           1, -- State,
           N'abcde'); -- First argument supplies the string.
-- The message text returned is: <<    abc>>.
GO

Examples

A. Returning error information from a CATCH block

The following code example shows how to use RAISERROR inside a TRY block to cause execution to jump to the associated CATCH block. It also shows how to use RAISERROR to return information about the error that invoked the CATCH block.

Note Note

RAISERROR only generates errors with state from 1 through 18. Because the PDW engine may raise errors with state 0, we recommend that you check the error state returned by ERROR_STATE before passing it as a value to the state parameter of RAISERROR.

Copy image Copy Code
BEGIN TRY
    -- RAISERROR with severity 11-18 will cause execution to
    -- jump to the CATCH block.
    RAISERROR ('Error raised in TRY block.', -- Message text.
               16, -- Severity.
               1 -- State.
               );
END TRY
BEGIN CATCH
    DECLARE @ErrorMessage NVARCHAR(4000);
    DECLARE @ErrorSeverity INT;
    DECLARE @ErrorState INT;

    SET @ErrorMessage = ERROR_MESSAGE();
    SET @ErrorSeverity = ERROR_SEVERITY();
    SET @ErrorState = ERROR_STATE();

    -- Use RAISERROR inside the CATCH block to return error
    -- information about the original error that caused
    -- execution to jump to the CATCH block.
    RAISERROR (@ErrorMessage, -- Message text.
               @ErrorSeverity, -- Severity.
               @ErrorState -- State.
               );
END CATCH;

B. Using a local variable to supply the message text

The following code example shows how to use a local variable to supply the message text for a RAISERROR statement.

Copy image Copy Code
DECLARE @StringVariable NVARCHAR(50);
SET @StringVariable = N'<<%7.3s>>';

RAISERROR (@StringVariable, -- Message text.
           10, -- Severity,
           1, -- State,
           N'abcde'); -- First argument supplies the string.
-- The message text returned is: <<    abc>>.
GO

See Also

Reference

PRINT (SQL Server PDW)
@@ERROR (SQL Server PDW)
TRY...CATCH (SQL Server PDW)
ERROR_MESSAGE (SQL Server PDW)
ERROR_NUMBER (SQL Server PDW)
ERROR_PROCEDURE (SQL Server PDW)
ERROR_SEVERITY (SQL Server PDW)
ERROR_STATE (SQL Server PDW)
THROW (SQL Server PDW)


SET @local_variable (SQL Server PDW)

[image]

SET @local_variable (SQL Server PDW)
See Also Send Feedback

Sets the value of a local variable in SQL Server PDW, previously created by using the DECLARE @local_variable statement.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET
{ @local_variable
    { = | += | -= | *= | /= | %= | &= | ^= | |= } expression
}

Arguments

@local_variable

Is the name of a variable of any type. Variable names must start with one at sign (@). Variable names must comply with the rules for identifiers which are explained in Object Naming Rules (SQL Server PDW).

{ += | -= | *= | /= | %= | &= | ^= | |= }

Compound assignment operator:

  1. += Add and assign

  2. -= Subtract and assign

  3. *= Multiply and assign

  4. /= Divide and assign

  5. %= Modulo and assign

  6. &= Bitwise AND and assign

  7. ^= Bitwise XOR and assign

  8. |= Bitwise OR and assign

expression

Is any valid expression. For more information, see Expressions (SQL Server PDW).

Examples

A. Printing the value of a variable initialized by using SET

The following example creates the @myvar variable, puts a string value into the variable, and prints the value of the @myvar variable.

Copy image Copy Code
DECLARE @myvar char(20);
SET @myvar = 'This is a test';
SELECT top 1 @myvar FROM sys.databases;

B. Using a local variable assigned a value by using SET in a SELECT statement

The following example creates a local variable named @dept and uses this local variable in a SELECT statement to find the first and last names of all employees who work in the Marketing department.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

DECLARE @dept char(25);
SET @dept = N'Marketing';
SELECT RTRIM(FirstName) + ' ' + RTRIM(LastName) AS Name
FROM DimEmployee
WHERE DepartmentName = @dept;

C. Using a compound assignment for a local variable

The following two examples produce the same result. They create a local variable named @NewBalance, multiplies it by 10 and displays the new value of the local variable in a SELECT statement. The second example uses a compound assignment operator.

Copy image Copy Code
/* Example one */
DECLARE  @NewBalance  int ;
SET  @NewBalance  =  10;
SET  @NewBalance  =  @NewBalance  *  10;
SELECT  TOP 1 @NewBalance FROM sys.tables;

/* Example Two */
DECLARE @NewBalance int = 10;
SET @NewBalance *= 10;
SELECT TOP 1 @NewBalance FROM sys.tables;

D. Assigning a value from a query

The following example uses a query to assign a value to a variable.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

DECLARE @rows int;
SET @rows = (SELECT COUNT(*) FROM dbo.DimCustomer);
SELECT TOP 1 @rows FROM sys.tables;

See Also


THROW (SQL Server PDW)

[image]

THROW (SQL Server PDW)
See Also Send Feedback

Raises user specified errors with a fixed severity of 16 and transfers execution to a CATCH block of a TRY…CATCH construct in Analytics Platform System. It is similar to RAISERROR statement. When used without additional arguments, THROW can be used to re-throw the same error within a CATCH block.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

THROW [ { error_number | @local_variable },
        { message | @local_variable },
        { state | @local_variable } ]
[ ; ]

Arguments

error_number

Is a constant or variable that represents the exception. error_number is int and must be between 50000 and 100000 or between 150000 and 2147483647.

message

Is an string or variable that describes the exception. message is nvarchar(2048).

state

Is a constant or variable between 0 and 255 that indicates the state to associate with the message. state is tinyint.

Remarks

The THROW statement must be executed as the first statement in the batch, or the statement before the THROW statement must be followed by the semicolon (;) statement terminator.

If a TRY…CATCH construct is not available, the session is ended. The procedure where the exception is raised is set. The severity is set to 16.

If the THROW statement is specified without parameters, it must appear inside a CATCH block. This causes the caught exception to be raised. Any error that occurs in a THROW statement causes the statement batch to be ended.

THROW in PDW differs from SQL Server in two ways:

  • Error numbers which fall in the PDW reserved range of 100,000 to 150,000 (inclusive) is not be supported.

  • SQL Server reserved error number range (0 – 50,000, inclusive) is not be supported, by default.

Differences Between RAISERROR and THROW

The following table lists differences between the RAISERROR and THROW statements.

RAISERROR statement | THROW statement
The msg_str parameter can contain printf formatting styles. | The message parameter does not accept printf style formatting.
The severity parameter specifies the severity of the exception. | There is no severity parameter. The exception severity is always set to 16.

Examples

A. Using THROW to raise an exception

The following example shows how to use the THROW statement to raise an exception.

Transact-SQL Copy image Copy Code
THROW 51000, 'The record does not exist.', 1;

Here is the result set.

Msg 51000, Level 16, State 1, Line 1

The record does not exist.

See Also

Reference

XACT_STATE (SQL Server PDW)
@@ERROR (SQL Server PDW)
TRY...CATCH (SQL Server PDW)
ERROR_MESSAGE (SQL Server PDW)
ERROR_NUMBER (SQL Server PDW)
ERROR_PROCEDURE (SQL Server PDW)
ERROR_SEVERITY (SQL Server PDW)
ERROR_STATE (SQL Server PDW)
RAISERROR (SQL Server PDW)


TRY...CATCH (SQL Server PDW)

[image]

TRY...CATCH (SQL Server PDW)
See Also Send Feedback

Implements error handling for Transact-SQL that is similar to the exception handling in the Microsoft Visual C# and Microsoft Visual C++ languages. A group of SQL statements can be enclosed in a TRY block. If an error occurs in the TRY block, control is passed to another group of statements that is enclosed in a CATCH block.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

BEGIN TRY
     { sql_statement | statement_block }
END TRY
BEGIN CATCH
     [ { sql_statement | statement_block } ]
END CATCH
[ ; ]

Arguments

sql_statement

Is any Transact-SQL statement.

statement_block

Any group of Transact-SQL statements in a batch or enclosed in a BEGIN…END block.

Remarks

A TRY…CATCH construct catches all execution errors that have a severity higher than 10.

A TRY block must be immediately followed by an associated CATCH block. Including any other statements between the END TRY and BEGIN CATCH statements generates a syntax error.

A TRY…CATCH construct cannot span multiple batches. A TRY…CATCH construct cannot span multiple blocks of Transact-SQL statements. For example, a TRY…CATCH construct cannot span two BEGIN…END blocks of Transact-SQL statements and cannot span an IF…ELSE construct.

If there are no errors in the code that is enclosed in a TRY block, when the last statement in the TRY block has finished running, control passes to the statement immediately after the associated END CATCH statement. If there is an error in the code that is enclosed in a TRY block, control passes to the first statement in the associated CATCH block. If the END CATCH statement is the last statement in a stored procedure, control is passed back to the statement that called the stored procedure.

When the code in the CATCH block finishes, control passes to the statement immediately after the END CATCH statement. Errors trapped by a CATCH block are not returned to the calling application. If any part of the error information must be returned to the application, the code in the CATCH block must do so by using mechanisms such as SELECT result sets or the RAISERROR and PRINT statements.

TRY…CATCH constructs can be nested. Either a TRY block or a CATCH block can contain nested TRY…CATCH constructs. For example, a CATCH block can contain an embedded TRY…CATCH construct to handle errors encountered by the CATCH code.

Errors encountered in a CATCH block are treated like errors generated anywhere else. If the CATCH block contains a nested TRY…CATCH construct, any error in the nested TRY block will pass control to the nested CATCH block. If there is no nested TRY…CATCH construct, the error is passed back to the caller.

TRY…CATCH constructs catch unhandled errors from stored procedures executed by the code in the TRY block. Alternatively, the stored procedures or triggers can contain their own TRY…CATCH constructs to handle errors generated by their code. For example, when a TRY block executes a stored procedure and an error occurs in the stored procedure, the error can be handled in the following ways:

  • If the stored procedure does not contain its own TRY…CATCH construct, the error returns control to the CATCH block associated with the TRY block that contains the EXECUTE statement.

  • If the stored procedure contains a TRY…CATCH construct, the error transfers control to the CATCH block in the stored procedure. When the CATCH block code finishes, control is passed back to the statement immediately after the EXECUTE statement that called the stored procedure.

Retrieving Error Information

In the scope of a CATCH block, the following system functions can be used to obtain information about the error that caused the CATCH block to be executed:

  • ERROR_NUMBER() returns the number of the error.

  • ERROR_SEVERITY() returns the severity.

  • ERROR_STATE() returns the error state number.

  • ERROR_PROCEDURE() returns the name of the stored procedure or trigger where the error occurred.

  • ERROR_MESSAGE() returns the complete text of the error message. The text includes the values supplied for any substitutable parameters, such as lengths, object names, or times.

These functions return NULL if they are called outside the scope of the CATCH block. Error information can be retrieved by using these functions from anywhere within the scope of the CATCH block. For example, the following script shows a stored procedure that contains error-handling functions. In the CATCH block of a TRY…CATCH construct, the stored procedure is called and information about the error is returned.

Copy image Copy Code
-- Verify that the stored procedure does not already exist.
IF OBJECT_ID ( 'usp_GetErrorInfo', 'P' ) IS NOT NULL
    DROP PROCEDURE usp_GetErrorInfo;
GO

-- Create procedure to retrieve error information.
CREATE PROCEDURE usp_GetErrorInfo
AS
SELECT
    ERROR_NUMBER() AS ErrorNumber
    ,ERROR_SEVERITY() AS ErrorSeverity
    ,ERROR_STATE() AS ErrorState
    ,ERROR_PROCEDURE() AS ErrorProcedure
    ,ERROR_MESSAGE() AS ErrorMessage;
GO

BEGIN TRY
    -- Generate divide-by-zero error.
    SELECT 1/0;
END TRY
BEGIN CATCH
    -- Execute error retrieval routine.
    EXECUTE usp_GetErrorInfo;
END CATCH;

Errors Unaffected by a TRY…CATCH Construct

TRY…CATCH constructs do not trap the following conditions:

  • Warnings or informational messages that have a severity of 10 or lower.

  • Attentions, such as client-interrupt requests or broken client connections.

  • When the session is ended by a system administrator by using the KILL statement.

The following types of errors are not handled by a CATCH block when they occur at the same level of execution as the TRY…CATCH construct:

  • Compile errors, such as syntax errors, that prevent a batch from running.

These errors are returned to the level that ran the batch, stored procedure, or trigger.

Some errors that are compile errors in SQL Server (such as binding errors) are execution errors in PDW and they will be caught in a CATCH block.

Transaction Behavior

The transactional behavior of TRY... CATCH blocks is not identical to that of the SQL Server. In SQL Server, when a CATCH block is entered, the transaction is uncommittable, but the data can be read. This state is characterized by the value of the XACT_STATE function of -1. While the state is -1, SELECT statements can be used to query data affected prior to an error being thrown.

In PDW, there is a new state for the active transaction, in which data can be neither committed, nor accessed. This state can be identified by invoking the XACT_STATE function where -2 is the expected value for that state. We call this transaction state rollback-only.

After entering a CATCH block and up until ROLLBACK is issued by the user or the end of batch is reached, no data-access statements are allowed in the batch (SELECT, DML, or DDL statements). The statements that can be executed within a transaction that reaches this state are SET, DECLARE, PRINT, RAISERROR, THROW and control-flow statements (IF/ELSE, WHILE, TRY…CATCH). This state can be identified by invoking the XACT_STATE function where -2 is the expected value for that state. The following example demonstrates the XACT_STATE behavior.

Step | Statement | XACT_STATE | Comments
1 | BEGIN TRAN | 0 | Beginning transaction
2 | BEGIN TRY | 1 | Transaction is active.
3 | SELECT 1/0 | 1 | Division by zero happens here.
 | ... | N/A | Control passed to CATCH block.
 | END TRY | N/A |
4 | BEGIN CATCH | -2 | Transaction is now in a rollback-only state
5 | DECLARE @err int = @@ERROR | -2 | DECLARE, SET, control flow statements are allowed.
6 | DECLARE @xact int = XACT_STATE() | -2 | We cache error and transaction states for later use.
6 | END CATCH | -2 |
7 | -- SELECT @err, @xact | -2 | SELECTs are still prohibited here.
8 | ROLLBACK | -2 | Rolling back the transaction.
9 | SELECT @err, @xact | 0 | No active transactions, SELECTs are allowed.

Since the transactional behavior of TRY.. CATCH blocks has been altered in PDW, and that change is manifested and can be detected by the altered value of XACT_STATE function in the catch block (-2 in PDW versus -1 in SQL Server), it would be a good practice for scripts which handle errors to check for negative values of XACT_STATE, instead of specifically checking for -1 or -2.

For more information about uncommittable transactions and the XACT_STATE function, see XACT_STATE (SQL Server PDW).

Examples

A. Using TRY…CATCH

The following example shows a SELECT statement that will generate a divide-by-zero error. The error causes execution to jump to the associated CATCH block.

Copy image Copy Code
BEGIN TRY
    -- Generate a divide-by-zero error.
    SELECT 1/0;
END TRY
BEGIN CATCH
    SELECT
        ERROR_NUMBER() AS ErrorNumber
        ,ERROR_SEVERITY() AS ErrorSeverity
        ,ERROR_STATE() AS ErrorState
        ,ERROR_PROCEDURE() AS ErrorProcedure
        ,ERROR_MESSAGE() AS ErrorMessage;
END CATCH;
GO

See Also

Reference

XACT_STATE (SQL Server PDW)
@@ERROR (SQL Server PDW)
ERROR_MESSAGE (SQL Server PDW)
ERROR_NUMBER (SQL Server PDW)
ERROR_PROCEDURE (SQL Server PDW)
ERROR_SEVERITY (SQL Server PDW)
ERROR_STATE (SQL Server PDW)
RAISERROR (SQL Server PDW)
THROW (SQL Server PDW)


WHILE (SQL Server PDW)

[image]

WHILE (SQL Server PDW)
See Also Send Feedback

Sets a condition for the repeated execution of a SQL statement or statement block. The statements are run repeatedly as long as the specified condition is true. The execution of statements in the WHILE loop can be controlled from inside the loop with the BREAK keyword.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

WHILE Boolean_expression
     { sql_statement | statement_block | BREAK }

Arguments

Boolean_expression

Is an expression that returns TRUE or FALSE. If the Boolean expression contains a SELECT statement, the SELECT statement must be enclosed in parentheses.

{sql_statement | statement_block}

Is any SQL statement or statement grouping as defined with a statement block. To define a statement block, use the control-of-flow keywords BEGIN and END.

BREAK

Causes an exit from the innermost WHILE loop. Any statements that appear after the END keyword, marking the end of the loop, are run.

General Remarks

If two or more WHILE loops are nested, the inner BREAK exits to the next outermost loop. All the statements after the end of the inner loop run first, and then the next outermost loop restarts.

Examples

Simple While Loop

In the following example, if the average list price of a product is less than $300, the WHILE loop doubles the prices and then selects the maximum price. If the maximum price is less than or equal to $500, the WHILE loop restarts and doubles the prices again. This loop continues doubling the prices until the maximum price is greater than $500, and then exits the WHILE loop.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
WHILE ( SELECT AVG(ListPrice) FROM dbo.DimProduct) < $300
BEGIN
    UPDATE dbo.DimProduct
        SET ListPrice = ListPrice * 2;
    SELECT MAX ( ListPrice) FROM dbo.DimProduct
    IF ( SELECT MAX (ListPrice) FROM dbo.DimProduct) > $500
        BREAK;
END

See Also


Transactions (SQL Server PDW)

[image]

Transactions (SQL Server PDW)
See Also Send Feedback

A transaction is a group of one or more database statements that are either wholly committed or wholly rolled back. Each transaction is atomic, consistent, isolated, and durable (ACID). If the transaction succeeds, all statements within it are committed. If the transaction fails, that is at least one of the statements in the group fails, then the entire group is rolled back..

The beginning and end of transactions depends on the AUTOCOMMIT setting and the BEGIN TRANSACTION, COMMIT, and ROLLBACK statements. SQL Server PDW supports the following types of transactions:

  • Explicit transactions start with the BEGIN TRANSACTION statement and end with the COMMIT or ROLLBACK statement.

  • Auto-commit transactions initiate automatically within a session and do not start with the BEGIN TRANSACTION statement. When the AUTOCOMMIT setting is ON, each statement runs in a transaction and no explicit COMMIT or ROLLBACK is necessary. When the AUTOCOMMIT setting is OFF, a COMMIT or ROLLBACK statement is required to determine the outcome of the transaction. In SQL Server PDW, autocommit transactions begin immediately after a COMMIT or ROLLBACK statement, or after a SET AUTOCOMMIT OFF statement.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

BEGIN TRANSACTION [;]
COMMIT [ TRAN | TRANSACTION | WORK ] [;]
ROLLBACK [ TRAN | TRANSACTION | WORK ] [;]
SET AUTOCOMMIT { ON | OFF } [;]
SET IMPLICIT_TRANSACTIONS { ON | OFF } [;]

Arguments

BEGIN TRANSACTION

Marks the starting point of an explicit transaction.

COMMIT [ WORK ]

Marks the end of an explicit or autocommit transaction. This statement causes the changes in the transaction to be permanently committed to the database. The statement COMMIT is identical to COMMIT WORK, COMMIT TRAN, and COMMIT TRANSACTION.

ROLLBACK [ WORK ]

Rolls back a transaction to the beginning of the transaction. No changes for the transaction are committed to the database. The statement ROLLBACK is identical to ROLLBACK WORK, ROLLBACK TRAN, and ROLLBACK TRANSACTION.

SET AUTOCOMMIT { ON | OFF }

Determines how transactions can start and end.

ON

Each statement runs under its own transaction and no explicit COMMIT or ROLLBACK statement is necessary. Explicit transactions are allowed when AUTOCOMMIT is ON.

OFF

SQL Server PDW automatically initiates a transaction when a transaction is not already in progress. Any subsequent statements are run as part of the transaction and a COMMIT or ROLLBACK is necessary to determine the outcome of the transaction. As soon as a transaction commits or rolls back under this mode of operation, the mode remains OFF, and SQL Server PDW initiates a new transaction. Explicit transactions are not allowed when AUTOCOMMIT is OFF.

If you change the AUTOCOMMIT setting within an active transaction, the setting does affect the current transaction and does not take affect until the transaction is completed.

If AUTOCOMMIT is ON, running another SET AUTOCOMMIT ON statement has no effect. Likewise, if AUTOCOMMIT is OFF, running another SET AUTOCOMMIT OFF has no effect.

SET IMPLICIT_TRANSACTIONS { ON | OFF }

This toggles the same modes as SET AUTOCOMMIT. When ON, SET IMPLICIT_TRANSACTIONS sets the connection into implicit transaction mode. When OFF, it returns the connection to autocommit mode. For more information, see SET IMPLICIT_TRANSACTIONS (SQL Server PDW)

Permissions

No specific permissions are required to run the transaction-related statements. Permissions are required to run the statements within the transaction.

Error Handling

If COMMIT or ROLLBACK are run and there is no active transaction, an error is raised.

If a BEGIN TRANSACTION is run while a transaction is already in progress, an error is raised. This can occur if a BEGIN TRANSACTION occurs after a successful BEGIN TRANSACTION statement or when the session is under SET AUTOCOMMIT OFF.

If an error other than a run-time statement error prevents the successful completion of an explicit transaction, SQL Server PDW automatically rolls back the transaction and frees all resources held by the transaction. For example, if the client's network connection to an instance of SQL Server PDW is broken or the client logs off the application, any uncommitted transactions for the connection are rolled back when the network notifies the instance of the break.

If a run-time statement error occurs in a batch, SQL Server PDW behaves consistent with SQL Server XACT_ABORT set to ON and the entire transaction is rolled back. For more information about the XACT_ABORT setting, see SET XACT_ABORT (Transact-SQL).

General Remarks

A session can only run one transaction at a given time; save points and nested transactions are not supported.

It is the responsibility of the SQL programmer to issue COMMIT only at a point when all data referenced by the transaction is logically correct.

When a session is terminated before a transaction completes, the transaction is rolled back.

Transaction modes are managed at the session level. For example, if one session begins an explicit transaction or sets AUTOCOMMIT to OFF, or sets IMPLICIT_TRANSACTIONS to ON, it has no effect on the transaction modes of any other session.

Limitations and Restrictions

You cannot roll back a transaction after a COMMIT statement is issued because the data modifications have been made a permanent part of the database.

The CREATE DATABASE and DROP DATABASE commands cannot be used inside an explicit transaction.

SQL Server PDW does not have a transaction sharing mechanism. This implies that at any given point in time, only one session can be doing work on any transaction in the system.

Locking Behavior

SQL Server PDW uses locking to ensure the integrity of transactions and maintain the consistency of databases when multiple users are accessing data at the same time. Locking is used by both implicit and explicit transactions. Each transaction requests locks of different types on the resources, such as tables or databases on which the transaction depends. All SQL Server PDW locks are table level or higher. The locks block other transactions from modifying the resources in a way that would cause problems for the transaction requesting the lock. Each transaction frees its locks when it no longer has a dependency on the locked resources; explicit transactions retain locks until the transaction completes when it is either committed or rolled back.

Examples

A. Using an explicit transaction

Copy image Copy Code
BEGIN TRANSACTION;
DELETE FROM HumanResources.JobCandidate
    WHERE JobCandidateID = 13;
COMMIT;

B. Rolling back a transaction

The following example shows the effect of rolling back a transaction. In this example, the ROLLBACK statement will roll back the INSERT statement, but the created table will still exist.

Copy image Copy Code
CREATE TABLE ValueTable (id int);
BEGIN TRANSACTION;
       INSERT INTO ValueTable VALUES(1);
       INSERT INTO ValueTable VALUES(2);
ROLLBACK;

C. Setting AUTOCOMMIT

The following example sets the AUTOCOMMIT setting to ON.

Copy image Copy Code
SET AUTOCOMMIT ON;

The following example sets the AUTOCOMMIT setting to OFF.

Copy image Copy Code
SET AUTOCOMMIT OFF;

D. Using an implicit multi-statement transaction

Copy image Copy Code
SET AUTOCOMMIT OFF;
CREATE TABLE ValueTable (id int);
INSERT INTO ValueTable VALUES(1);
INSERT INTO ValueTable VALUES(2);
COMMIT;

See Also

Reference

SET IMPLICIT_TRANSACTIONS (SQL Server PDW)
SET TRANSACTION ISOLATION LEVEL (SQL Server PDW)
@@TRANCOUNT (SQL Server PDW)


CREATE DIAGNOSTICS SESSION (SQL Server PDW)

[image]

CREATE DIAGNOSTICS SESSION (SQL Server PDW)
See Also Send Feedback

Diagnostics sessions allow you to save detailed, user-defined diagnostic information on system or query performance.

Diagnostics sessions are typically used to debug performance for a specific query, or to monitor the behavior of a specific appliance component during appliance operation.

Note Note

You should be familiar with XML in order to use diagnostics sessions.

Syntax

Creating a new diagnostics session:
CREATE DIAGNOSTICS SESSION diagnostics_name AS N’{<session_xml>}’;

<session_xml>::
<Session>
   [ <MaxItemCount>max_item_count_num</MaxItemCount> ]
   <Filter>
      { <Event Name=”event_name”/>
         [ <Where><filter_property_name Name=”value” ComparisonType="comp_type"/></Where> ] [ ,...n ]
      } [ ,...n ]
   </Filter> ]
   <Capture>
      <Property Name=”property_name”/> [ ,...n ]
   </Capture>
<Session>

Retrieving results for a diagnostics session:
SELECT * FROM master.sysdiag.diagnostics_name ;

Removing results for a diagnostics session:
DROP DIAGNOSTICS SESSION diagnostics_name ;

Arguments

diagnostics_name

The name of the diagnostics session. Diagnostics session names can include characters a-z, A-Z, and 0-9 only. Also, diagnostics session names must start with a character. diagnostics_name is limited to 127 characters.

max_item_count_num

The number of events to be persisted in a view. For example, if 100 is specified, the 100 most recent events matching the filter criteria will be persisted to the diagnostics session. If fewer than 100 matching events are found, the diagnostics session will contain less than 100 events. max_item_count_num must be at least 100 and less than or equal to 100,000.

event_name

Defines the actual events to be collected in the diagnostics session. event_name is one of the events listed in sys.pdw_diag_events where sys.pdw_diag_events.is_enabled='True'.

filter_property_name

The name of the property on which to restrict results. For example, if you want to limit based on session id, filter_property_name should be SessionId. See property_name below for a list of potential values for filter_property_name.

value

A value to evaluate against filter_property_name. The value type must match the property type. For example, if the property type is decimal, the type of value must be decimal.

comp_type

The comparison type. Potential values are:Equals, EqualsOrGreaterThan, EqualsOrLessThan, GreaterThan, LessThan, NotEquals, Contains, RegEx

property_name

A property related to the event. Property names can be part of the capture tag, or used as part of filtering criteria.

Property Name

Description

UserName

A user (login) name.

SessionId

A session ID.

QueryId

A query ID.

CommandType

A command type.

CommandText

Text within a command processed.

OperationType

The operation type for the event.

Duration

The duration of the event.

SPID

The Service Process ID.

Remarks

Each user is allowed a maximum of 10 concurrent diagnostics sessions. See sys.pdw_diag_sessions for a list of your current sessions, and drop any unneeded sessions using DROP DIAGNOSTICS SESSION.

Diagnostics sessions will continue to collect metadata until dropped.

Permissions

Requires the ALTER SERVER STATE permission.

Locking

Takes a shared lock on the Diagnostic Sessions table.

Examples

A. Creating a diagnostics session

This example creates a diagnostics session to record metrics of the database engine performance. The example creates a diagnostics session that listens for Engine Query running/end events and a blocking DMS event. What is returned is the command text, machine name, request id (query id) and the session that the event was created on.

Copy image Copy Code
CREATE DIAGNOSTICS SESSION MYDIAGSESSION AS N'
<Session>
   <MaxItemCount>100</MaxItemCount>
   <Filter>
      <Event Name="EngineInstrumentation:EngineQueryRunningEvent" />
      <Event Name="DmsCoreInstrumentation:DmsBlockingQueueEnqueueBeginEvent" />
      <Where>
         <SessionId Value="381" ComparisonType="NotEquals" />
      </Where>
   </Filter>
   <Capture>
      <Property Name="Query.CommandText" />
      <Property Name="MachineName" />
      <Property Name="Query.QueryId" />
      <Property Name="Alias" />
      <Property Name="Duration" />
      <Property Name="Session.SessionId" />
   </Capture>
</Session>';

After creation of the diagnostics session, run a query.

Copy image Copy Code
SELECT COUNT(EmployeeKey) FROM AdventureWorksPDW2012..FactSalesQuota;

Then view the diagnostics session results by selecting from the sysdiag schema.

Copy image Copy Code
SELECT * FROM master.sysdiag.MYDIAGSESSION;

Notice that the sysdiag schema contains a view that is named your diagnostics session name.

To see only the activity for your connection, add the Session.SPID property and add WHERE [Session.SPID] = @@spid; to the query.

When you are finished with the diagnostics session, drop it using the DROP DIAGNOSTICS command.

Copy image Copy Code
DROP DIAGNOSTICS SESSION MYDIAGSESSION;

B. Alternative diagnostic session

A second example with slightly different properties.

Copy image Copy Code
-- Determine the session_id of your current session
SELECT TOP 1 session_id();
-- Replace <*session_number*> in the code below with the numbers in your session_id
CREATE DIAGNOSTICS SESSION PdwOptimizationDiagnostics AS N'
<Session>
   <MaxItemCount>100</MaxItemCount>
   <Filter>
      <Event Name="EngineInstrumentation:MemoGenerationBeginEvent" />
      <Event Name="EngineInstrumentation:MemoGenerationEndEvent" />
      <Event Name="DSQLInstrumentation:OptimizationBeginEvent" />
      <Event Name="DSQLInstrumentation:OptimizationEndEvent" />
      <Event Name="DSQLInstrumentation:BuildRelOpContextTreeBeginEvent" />
      <Event Name="DSQLInstrumentation:PostPlanGenModifiersEndEvent" />
      <Where>
         <SessionId Value="<*session_number*>" ComparisonType="Equals" />
      </Where>
   </Filter>
   <Capture>
      <Property Name="Session.SessionId" />
      <Property Name="Query.QueryId" />
      <Property Name="Query.CommandText" />
      <Property Name="Name" />
      <Property Name="DateTimePublished" />
      <Property Name="DateTimePublished.Ticks" />
  </Capture>
</Session>';

Run a query, such as:

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
GO
SELECT * FROM dbo.FactFinance;

The following query returns the authorization timing:

Copy image Copy Code
SELECT *
FROM master.sysdiag.PdwOptimizationDiagnostics
ORDER BY DateTimePublished;

When you are finished with the diagnostics session, drop it using the DROP DIAGNOSTICS command.

Copy image Copy Code
DROP DIAGNOSTICS SESSION PdwOptimizationDiagnostics;

See Also

Reference

System Views (SQL Server PDW)


Procedures (SQL Server PDW)

[image]

Procedures (SQL Server PDW)
See Also Send Feedback

SQL Server PDW provides built-in procedures that you can use to perform operations related to database roles. SQL Server PDW includes the following system procedures:

sp_addrolemember (SQL Server PDW) | sp_pdw_remove_network_credentials (SQL Server PDW)
sp_columns (SQL Server PDW) | sp_pkeys (SQL Server PDW)
sp_datatype_info_90 (SQL Server PDW) | sp_prepare (SQL Server PDW)
sp_droprolemember (SQL Server PDW) | sp_special_columns_100 (SQL Server PDW)
sp_execute (SQL Server PDW) | sp_sproc_columns (SQL Server PDW)
sp_executesql (SQL Server PDW) | sp_statistics (SQL Server PDW)
sp_fkeys (SQL Server PDW) | sp_tables (SQL Server PDW)
sp_pdw_add_network_credentials (SQL Server PDW) | sp_unprepare (SQL Server PDW)
Note Note

Some additional system stored procedures are used only within an instance of SQL Server or through client APIs and are not intended for general customer use. These procedures are listed at System Stored Procedures (Transact-SQL). These procedures are subject to change and compatibility is not guaranteed. All procedures on the list are not available in SQL Server PDW.

See Also

Reference

Functions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Types (SQL Server PDW)


sp_addrolemember (SQL Server PDW)

[image]

sp_addrolemember (SQL Server PDW)
See Also Send Feedback

Adds a database user or a database role to a database role in the current database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_addrolemember 'role', 'security_account'

Arguments

'role'

Is the name of the database role in the current database. role is a sysname, with no default.

'security_account'

Is the security account being added to the role. security_account is a sysname, with no default. security_account can be a database user or a database role.

Return Code Values

0 (success) or 1 (failure)

Remarks

A member added to a role by using sp_addrolemember inherits the permissions of the role. Always check that the login exists and has access to the database.

A role cannot include itself as a member. Such "circular" definitions are not valid, even when membership is only indirectly implied by one or more intermediate memberships.

sp_addrolemember cannot add a fixed database role, fixed server role, or dbo to a role. sp_addrolemember cannot be executed within a user-defined transaction.

Permissions

Adding members to database user-defined roles requires one of the following:

  • Membership in the db_securityadmin or db_owner fixed database role.

  • Membership in the role that owns the role.

  • ALTER ANY ROLE permission or ALTER permission on the role.

Adding members to fixed database roles requires membership in the db_owner fixed database role.

Locking Behavior

Requires a Shared lock on the database.

Examples

A. Adding a Windows login

The following example adds the login LoginMary to the AdventureWorks2008R2 database as user UserMary. The user UserMary is then added to the Production role.

Note Note

Because the login LoginMary is known as the database user UserMary in the AdventureWorks2012 database, the user name UserMary must be specified. The statement will fail unless a Mary5 login exists. Logins and users usually have the same name. This example uses different names to differentiate the actions affecting the login vs. the user.

Copy image Copy Code
USE AdventureWorks2008R2;
GO
CREATE USER UserMary FOR LOGIN LoginMary ;
GO
EXEC sp_addrolemember 'Production', 'UserMary'

B. Adding a database user

The following example adds the database user UserMary to the Production database role in the current database.

Copy image Copy Code
EXEC sp_addrolemember 'Production', 'UserMary'

See Also

Reference

sp_droprolemember (SQL Server PDW)
CREATE ROLE (SQL Server PDW)
ALTER ROLE (SQL Server PDW)
DROP ROLE (SQL Server PDW)


sp_columns (SQL Server PDW)

[image]

sp_columns (SQL Server PDW)
See Also Send Feedback

Returns column information for the specified objects that can be queried in the current environment.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_columns [ @table_name = ] object
     [ , [ @table_owner = ] owner ]
     [ , [ @table_qualifier = ] qualifier ]
     [ , [ @column_name = ] column ]
     [ , [ @ODBCVer = ] ODBCVer ]

Arguments

[ @table_name=] object

Is the name of the object that is used to return catalog information. object can be a table, view, or other object that has columns such as table-valued functions. object is nvarchar(384), with no default. Wildcard pattern matching is supported.

[ @table_owner=] owner

Is the object owner of the object that is used to return catalog information. owner is nvarchar(384), with a default of NULL. Wildcard pattern matching is supported. If owner is not specified, the default object visibility rules of the underlying DBMS apply.

If the current user owns an object with the specified name, the columns of that object are returned. If owner is not specified and the current user does not own an object with the specified object, sp_columns looks for an object with the specified object owned by the database owner. If one exists, that object's columns are returned.

[ @table_qualifier=] qualifier

Is the name of the object qualifier. qualifier is sysname, with a default of NULL. Various DBMS products support three-part naming for objects (qualifier.owner.name). In SQL Server, this column represents the database name. In some products, it represents the server name of the object's database environment.

[ @column_name=] column

Is a single column and is used when only one column of catalog information is wanted. column is nvarchar(384), with a default of NULL. If column is not specified, all columns are returned. In SQL Server, column represents the column name as listed in the syscolumns table. Wildcard pattern matching is supported. For maximum interoperability, the gateway client should assume only SQL-92 standard pattern matching (the % and _ wildcard characters).

[ @ODBCVer=] ODBCVer

Is the version of ODBC that is being used. ODBCVer is int, with a default of 2. This indicates ODBC Version 2. Valid values are 2 or 3. For the behavior differences between versions 2 and 3, see the ODBC SQLColumns specification.

Return Code Values

None

Result Sets

The sp_columns catalog stored procedure is equivalent to SQLColumns in ODBC. The results returned are ordered by TABLE_QUALIFIER, TABLE_OWNER, and TABLE_NAME.

Column name | Data type | Description
TABLE_QUALIFIER | sysname | Object qualifier name. This field can be NULL.
TABLE_OWNER | sysname | Object owner name. This field always returns a value.
TABLE_NAME | sysname | Object name. This field always returns a value.
COLUMN_NAME | sysname | Column name, for each column of the TABLE_NAME returned. This field always returns a value.
DATA_TYPE | smallint | Integer code for ODBC data type. If this is a data type that cannot be mapped to an ODBC type, it is NULL. The native data type name is returned in the TYPE_NAME column.
TYPE_NAME | sysname | String representing a data type. The underlying DBMS presents this data type name.
PRECISION | int | Number of significant digits. The return value for the PRECISION column is in base 10.
LENGTH | int | Transfer size of the data.1
SCALE | smallint | Number of digits to the right of the decimal point.
RADIX | smallint | Base for numeric data types.
NULLABLE | smallint | Specifies nullability. 1 = NULL is possible. 0 = NOT NULL.
REMARKS | varchar(254) | This field always returns NULL.
COLUMN_DEF | nvarchar(4000) | Default value of the column.
SQL_DATA_TYPE | smallint | Value of the SQL data type as it appears in the TYPE field of the descriptor. This column is the same as the DATA_TYPE column, except for the datetime and SQL-92 interval data types. This column always returns a value.
SQL_DATETIME_SUB | smallint | Subtype code for datetime and SQL-92 interval data types. For other data types, this column returns NULL.
CHAR_OCTET_LENGTH | int | Maximum length in bytes of a character or integer data type column. For all other data types, this column returns NULL.
ORDINAL_POSITION | int | Ordinal position of the column in the object. The first column in the object is 1. This column always returns a value.
IS_NULLABLE | varchar(254) | Nullability of the column in the object. ISO rules are followed to determine nullability. An ISO SQL-compliant DBMS cannot return an empty string. YES = Column can include NULLS. NO = Column cannot include NULLS. This column returns a zero-length string if nullability is unknown. The value returned for this column is different from the value returned for the NULLABLE column.
SS_DATA_TYPE | tinyint | SQL Server data type used by extended stored procedures. For more information, see Data Types (SQL Server PDW).

1 For more information, see the Microsoft ODBC documentation.

Permissions

Requires SELECT permission on the schema.

Remarks

sp_columns follows the requirements for delimited identifiers. For more information, see Database Identifiers.

Examples

The following example returns column information for a specified table.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
EXEC sp_columns @table_name = N'DimEmployee',
   @table_owner = N'dbo';

See Also

Reference

sp_tables (SQL Server PDW)
Procedures (SQL Server PDW)


sp_configure (SQL Server PDW)

[image]

sp_configure (SQL Server PDW)
Send Feedback

Displays or changes global configuration settings for SQL Server PDW. Only one configuration change (hadoop connectivity) is currently supported by SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

--List all of the configuration options
sp_configure
[;]

--Configure Hadoop connectivity
sp_configure [ @configname = ] 'hadoop connectivity',
             [ @configvalue = ] { 0 | 1 | 2 | 3 | 4 | 5 }
[;]
RECONFIGURE
[;]

Arguments

[ @configname= ] 'option_name'

Is the name of a configuration option. option_name is varchar(35), with a default of NULL. If not specified, the complete list of options is returned.

[ @configvalue= ] 'value'

Is the new configuration setting. value is int, with a default of NULL. The maximum value depends on the individual option.

'hadoop connectivity'

Specifies the type of Hadoop data source for all connections from SQL Server PDW to Hadoop clusters or Microsoft Azure blob storage (WASB). This setting is required in order to create an external data source for an external table. For more information, see CREATE EXTERNAL DATA SOURCE (SQL Server PDW),

These are the Hadoop connectivity settings and their corresponding supported Hadoop data sources. Only one setting can be in effect at a time. Options 1 and 4 allow multiple types of external data sources to be created and used.

  • Option 0: Disable Hadoop connectivity

  • Option 1: Hortonworks (HDP 1.3) for Microsoft Server

  • Option 1: HDInsight on Analytics Platform System (version AU1) (HDP 1.3)

  • Option 1: Azure blob storage on Microsoft Azure (WASB[S]) (only applies to AU1)

  • Option 2: Hortonworks (HDP 1.3) for Linux

  • Option 3: Cloudera CDH 4.3 for Linux

  • Option 4: Hortonworks (HDP 2.0) for Windows Server

  • Option 4: HDInsight on Analytics Platform System (version AU2) (HDP 2.0)

  • Option 4: Azure blob storage on Microsoft Azure (WASB[S])

  • Option 5: Hortonworks (HDP 2.0) for Linux

For the complete instructions on configuring Hadoop connectivity, including when and how to restart the PDW region, see Configure PolyBase Connectivity to External Data (Analytics Platform System).

RECONFIGURE

Updates the run value (run_value) to match the configuration value (config_value). See Result Setsfor definitions of run_value and config_value. The new configuration value that is set by sp_configure does not become effective until the run value is set by the RECONFIGURE statement. After running RECONFIGURE, some changes require a restart of the PDW region.

Return Code Values

0 (success) or 1 (failure)

Result Sets

When executed with no parameters, sp_configure returns a result set with five columns.

Column name | Data type | Description
name | nvarchar(35) | Name of the configuration option.
minimum | int | Minimum value of the configuration option.
maximum | int | Maximum value of the configuration option.
config_value | int | Value that was set using sp_configure.
run_value | int | Current value in use by PDW. This value is set by running RECONFIGURE. The config_value and run_value are usually the same unless the value is in the process of being changed. A restart might be required before this run value is accurate, if the reconfiguration is in progress.

General Remarks

Configuration Settings that Require a Restart

Some changes require a restart of the PDW region before the run value will take effect. To restart the PDW region, Launch the Configuration Manager (Analytics Platform System) and follow the on-screen instructions.

These configuration settings require a restart after running RECONFIGURE.

  • 'hadoop connectivity'

Configuration Settings for Internal Use

These configuration names are for internal use only and should only be changed by Microsoft customer support.

  • 'redistribute mode'. The config_value is 0 by default. It changes to 1 when Microsoft customer support is re-distributing your data after adding scale units to the PDW region in the appliance.

Limitations and Restrictions

RECONFIGURE is not allowed in an explicit or implicit transaction.

Permissions

All users can execute sp_configure with no parameters or the @configname parameter.

Requires ALTER SETTINGS server-level permission or membership in the sysadmin fixed server role to change a configuration value or to run RECONFIGURE.

Examples

A. List all available configuration settings

The following example shows how to list all configuration options.

Copy image Copy Code
EXEC sp_configure;

The result returns the option name followed by the minimum and maximum values for the option. The config_value is the value that SQL Server PDW will use when reconfiguration is complete. The run_value is the value that is currently being used. The config_value and run_value are usually the same unless the value is in the process of being changed.

B. List the configuration settings for one configuration name

Copy image Copy Code
EXEC sp_configure @configname='hadoop connectivity';

C. Set Hadoop connectivity

Setting Hadoop connectivity requires a few more steps in addition to running sp_configure. For the full procedure, see Configure PolyBase Connectivity to External Data (Analytics Platform System)


sp_datatype_info_90 (SQL Server PDW)

[image]

sp_datatype_info_90 (SQL Server PDW)
See Also Send Feedback

Returns information about the data types supported by the current environment.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_datatype_info_90 [ [ @data_type = ] data_type ]
     [ , [ @ODBCVer = ] odbc_version ]

Arguments

[ @data_type= ] data_type

Is the code number for the specified data type. To obtain a list of all data types, omit this parameter. data_type is int, with a default of 0.

[ @ODBCVer= ] odbc_version

Is the version of ODBC that is used. odbc_version is tinyint, with a default of 2.

Return Code Values

None

Result Sets

Column name | Data type | Description
TYPE_NAME | sysname | DBMS-dependent data type.
DATA_TYPE | smallint | Code for the ODBC type to which all columns of this type are mapped.
PRECISION | int | Maximum precision of the data type on the data source. NULL is returned for data types for which precision is not applicable. The return value for the PRECISION column is in base 10.
LITERAL_PREFIX | varchar(32) | Character or characters used before a constant. For example, a single quotation mark (') for character types and 0x for binary.
LITERAL_SUFFIX | varchar(32) | Character or characters used to terminate a constant. For example, a single quotation mark (') for character types and no quotation marks for binary.
CREATE_PARAMS | varchar(32) | Description of the creation parameters for this data type. For example, decimal is "precision, scale", float is NULL, and varchar is "max_length".
NULLABLE | smallint | Specifies nullability. 1 = Allows null values. 0 = Does not allow null values.
CASE_SENSITIVE | smallint | Specifies case sensitivity. 1 = All columns of this type are case-sensitive (for collations). 0 = All columns of this type are case-insensitive.
SEARCHABLE | smallint | Specifies the search capability of the column type: 1 = Cannot be searched. 2 = Searchable with LIKE. 3 = Searchable with WHERE. 4 = Searchable with WHERE or LIKE.
UNSIGNED_ATTRIBUTE | smallint | Specifies the sign of the data type. 1 = Data type unsigned. 0 = Data type signed.
MONEY | smallint | Specifies the money data type. 1 = money data type. 0 = Not a money data type.
AUTO_INCREMENT | smallint | Specifies autoincrementing. 1 = Autoincrementing. 0 = Not autoincrementing. NULL = Attribute not applicable. An application can insert values into a column that has this attribute, but the application cannot update the values in the column. With the exception of the bit data type, AUTO_INCREMENT is valid only for data types that belong to the Exact Numeric and Approximate Numeric data type categories.
LOCAL_TYPE_NAME | sysname | Localized version of the data source-dependent name of the data type. For example, DECIMAL is DECIMALE in French. NULL is returned if a localized name is not supported by the data source.
MINIMUM_SCALE | smallint | Minimum scale of the data type on the data source. If a data type has a fixed scale, the MINIMUM_SCALE and MAXIMUM_SCALE columns both contain this value. NULL is returned where scale is not applicable.
MAXIMUM_SCALE | smallint | Maximum scale of the data type on the data source. If the maximum scale is not defined separately on the data source, but is instead defined to be the same as the maximum precision, this column contains the same value as the PRECISION column.
SQL_DATA_TYPE | smallint | Value of the SQL data type as it appears in the TYPE field of the descriptor. This column is the same as the DATA_TYPE column, except for the datetime and ANSI interval data types. This field always returns a value.
SQL_DATETIME_SUB | smallint | datetime or ANSI interval subcode if the value of SQL_DATA_TYPE is SQL_DATETIME or SQL_INTERVAL. For data types other than datetime and ANSI interval, this field is NULL.
NUM_PREC_RADIX | int | Number of bits or digits for calculating the maximum number that a column can hold. If the data type is an approximate numeric data type, this column contains the value 2 to indicate several bits. For exact numeric types, this column contains the value 10 to indicate several decimal digits. Otherwise, this column is NULL. By combining the precision with radix, the application can calculate the maximum number that the column can hold.
INTERVAL_PRECISION | smallint | Value of interval leading precision if data_type is interval; otherwise NULL.
USERTYPE | smallint | usertype value from the systypes table.

Remarks

sp_datatype_info is equivalent to SQLGetTypeInfo in ODBC. The results returned are ordered by DATA_TYPE and then by how closely the data type maps to the corresponding ODBC SQL data type.

Permissions

Requires membership in the public role.

Examples

The following example retrieves information for the sysname and nvarchar data types by specifying the data_type value of -9.

Copy image Copy Code
USE master;
GO
EXEC sp_datatype_info_90 -9;
GO

See Also

Reference

Procedures (SQL Server PDW)

Concepts

Data Types (SQL Server PDW)


sp_droprolemember (SQL Server PDW)

[image]

sp_droprolemember (SQL Server PDW)
See Also Send Feedback

Removes a security account from a SQL Server role in the current database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_droprolemember 'role' , 'security_account'

Arguments

'role'

Is the name of the role from which the member is being removed. role is sysname, with no default. role must exist in the current database.

'security_account'

Is the name of the security account being removed from the role. security_account is sysname, with no default. security_account can be a database user or another database role. security_account must exist in the current database.

Return Code Values

0 (success) or 1 (failure)

Remarks

sp_droprolemember removes a member from a database role by deleting a row from the sysmembers table. When a member is removed from a role the member loses any permissions it has by membership in that role.

Users cannot be removed from the public role, and dbo cannot be removed from any role.

sp_droprolemember cannot be executed within a user-defined transaction.

Query sys.database_role_members (SQL Server PDW) to see the members of a database role. Use sp_addrolemember to add a member to a role.

Permissions

Requires ALTER permission on the role.

Locking Behavior

Requires a Shared lock on the database.

Examples

The following example removes the user JonB from the role Sales.

Copy image Copy Code
EXEC sp_droprolemember 'Sales', 'JonB'

See Also

Reference

sp_addrolemember (SQL Server PDW)
DROP ROLE (SQL Server PDW)


sp_execute (SQL Server PDW)

[image]

sp_execute (SQL Server PDW)
See Also Send Feedback

Executes a prepared Transact-SQL statement using a specified handle and optional parameter value. sp_execute is invoked by specifying ID =12 in a tabular data stream (TDS) packet.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_execute handle OUTPUT
    [,bound_param  ]  [,...n ]  ]

Arguments

handle

Is the handle value returned by sp_prepare. handle is a required parameter that calls for int input value.

bound_param

Signifies the use of additional parameters. bound_param is a required parameter that calls for input values of any data type to signify additional parameters for the procedure.

Note Note

bound_param must match the declarations made by the sp_prepare params value and can be in the form @name = value or value.

See Also

Reference

Procedures (SQL Server PDW)
sp_prepare (SQL Server PDW)
sp_unprepare (SQL Server PDW)


sp_executesql (SQL Server PDW)

[image]

sp_executesql (SQL Server PDW)
See Also Send Feedback

Executes a Transact-SQL statement or batch that can be reused many times, or one that has been built dynamically. The Transact-SQL statement or batch can contain embedded parameters.

Security note Security Note

Run time-compiled Transact-SQL statements can expose applications to malicious attacks.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_executesql [ @stmt = ] statement
[
  { , [ @params = ] N'@parameter_name data_type [ OUT | OUTPUT ][ ,...n ]' }
     { , [ @param1 = ] 'value1' [ ,...n ] }
]

Arguments

[ @stmt= ] statement

Is a Unicode string that contains a Transact-SQL statement or batch. @stmt must be either a Unicode constant or a Unicode variable. More complex Unicode expressions, such as concatenating two strings with the + operator, are not allowed. Character constants are not allowed. If a Unicode constant is specified, it must be prefixed with an N. For example, the Unicode constant N'sp_who' is valid, but the character constant 'sp_who' is not. If specified as a string literal, @stmt has a maximum size of 2 GB, the maximum size of nvarchar(max). If specified as a Unicode variable, @stmt cannot exceed 4000 characters due to SQL Server PDW limitations related to Unicode variables.

Note Note

@stmt can contain parameters having the same form as a variable name, for example: N'SELECT * FROM HumanResources.Employee WHERE EmployeeID = @IDParameter'

Each parameter included in @stmt must have a corresponding entry in both the @params parameter definition list and the parameter values list.

[ @params= ] N'@parameter_namedata_type [ ,... n ] '

Is one string that contains the definitions of all parameters that have been embedded in @stmt. The string must be either a Unicode constant or a Unicode variable. Each parameter definition consists of a parameter name and a data type. n is a placeholder that indicates additional parameter definitions. Every parameter specified in @stmtmust be defined in @params. If the Transact-SQL statement or batch in @stmt does not contain parameters, @params is not required. The default value for this parameter is NULL. If specified as a string literal, @params has a maximum size of 2 GB, the maximum size of nvarchar(max). If specified as a Unicode variable, @params cannot exceed 4000 characters due to SQL Server PDW limitations related to Unicode variables.

[ @param1= ] 'value1'

Is a value for the first parameter that is defined in the parameter string. The value can be a Unicode constant or a Unicode variable. There must be a parameter value supplied for every parameter included in @stmt. The values are not required when the Transact-SQL statement or batch in @stmt has no parameters. If specified as a string literal, @param1 has a maximum size of 2 GB, the maximum size of nvarchar(max). If specified as a Unicode variable, @param1 cannot exceed 4000 characters due to SQL Server PDW limitations related to Unicode variables.

[ OUT | OUTPUT ]

Indicates that the parameter is an output parameter. text, ntext, and image parameters can be used as OUTPUT parameters, unless the procedure is a common language runtime (CLR) procedure. An output parameter that uses the OUTPUT keyword can be a cursor placeholder, unless the procedure is a CLR procedure.

n

Is a placeholder for the values of additional parameters. Values can only be constants or variables. Values cannot be more complex expressions such as functions, or expressions built by using operators.

Return Code Values

None

Result Sets

Returns the result sets from all the SQL statements built into the SQL string.

Remarks

sp_executesql parameters must be entered in the specific order as described in the "Syntax" section earlier in this topic. If the parameters are entered out of order, an error message will occur.

sp_executesql has the same behavior as EXECUTE with regard to batches, the scope of names, and database context. The Transact-SQL statement or batch in the sp_executesql @stmt parameter is not compiled until the sp_executesql statement is executed. The contents of @stmt are then compiled and executed as an execution plan separate from the execution plan of the batch that called sp_executesql. The sp_executesql batch cannot reference variables declared in the batch that calls sp_executesql. Local cursors or variables in the sp_executesql batch are not visible to the batch that calls sp_executesql. Changes in database context last only to the end of the sp_executesql statement.

sp_executesql can be used instead of stored procedures to execute a Transact-SQL statement many times when the change in parameter values to the statement is the only variation. Because the Transact-SQL statement itself remains constant and only the parameter values change, the SQL Server query optimizer is likely to reuse the execution plan it generates for the first execution.

Note Note

To improve performance use fully qualified object names in the statement string.

sp_executesql supports the setting of parameter values separately from the Transact-SQL string as shown in the following example.

Copy image Copy Code
USE AdventureWorksPDW2012
GO

DECLARE @IntVariable int;
DECLARE @SQLString nvarchar(500);
DECLARE @ParmDefinition nvarchar(500);

/* Build the SQL string one time.*/

SET @SQLString =
     N'SELECT EmployeeKey, LastName, FirstName, Title, LoginID
       FROM AdventureWorksPDW2012.dbo.DimEmployee
       WHERE EmployeeKey = @EmployeeKey';
SET @ParmDefinition = N'@EmployeeKey tinyint';
/* Execute the string with the first parameter value. */
SET @IntVariable = 197;
EXECUTE sp_executesql @SQLString, @ParmDefinition,
                      @EmployeeKey = @IntVariable;
/* Execute the same string with the second parameter value. */
SET @IntVariable = 109;
EXECUTE sp_executesql @SQLString, @ParmDefinition,
                      @EmployeeKey = @IntVariable;

Being able to substitute parameters in sp_executesql offers the following advantages to using the EXECUTE statement to execute a string:

  • Because the actual text of the Transact-SQL statement in the sp_executesql string does not change between executions, the query optimizer will probably match the Transact-SQL statement in the second execution with the execution plan generated for the first execution. Therefore, SQL Server does not have to compile the second statement.

  • The Transact-SQL string is built only one time.

  • The integer parameter is specified in its native format. Casting to Unicode is not required.

Permissions

Requires membership in the public role.

Examples

A. Executing a simple SELECT statement

The following example creates and executes a simple SELECT statement that contains an embedded parameter named @level.

Copy image Copy Code
USE AdventureWorksPDW2012
GO

EXECUTE sp_executesql
          N'SELECT * FROM AdventureWorksPDW2012.dbo.DimEmployee
          WHERE EmployeeKey = @level',
          N'@level tinyint',
          @level = 109;

For additional examples, see sp_executesql (Transact-SQL).

See Also

Reference

EXECUTE (SQL Server PDW)
Procedures (SQL Server PDW)


sp_fkeys (SQL Server PDW)

[image]

sp_fkeys (SQL Server PDW)
See Also Send Feedback

SQL Server PDW does not support foreign keys, but it supports this procedure to allow integration of tools and applications. Returns logical foreign key information for the current database. This procedure shows foreign key relationships including disabled foreign keys.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_fkeys [ @pktable_name = ] 'pktable_name'
     [ , [ @pktable_owner = ] 'pktable_owner' ]
     [ , [ @pktable_qualifier = ] 'pktable_qualifier' ]
     { , [ @fktable_name = ] 'fktable_name' }
     [ , [ @fktable_owner = ] 'fktable_owner' ]
     [ , [ @fktable_qualifier = ] 'fktable_qualifier' ]

Arguments

[ @pktable_name=] 'pktable_name'

Is the name of the table, with the primary key, used to return catalog information. pktable_name is sysname, with a default of NULL. Wildcard pattern matching is not supported. This parameter or the fktable_name parameter, or both, must be supplied.

[ @pktable_owner=] 'pktable_owner'

Is the name of the owner of the table (with the primary key) used to return catalog information. pktable_owner is sysname, with a default of NULL. Wildcard pattern matching is not supported. If pktable_owner is not specified, the default table visibility rules of the underlying DBMS apply.

If the current user owns a table with the specified name, that table's columns are returned. If pktable_owner is not specified and the current user does not own a table with the specified pktable_name, the procedure looks for a table with the specified pktable_name owned by the database owner. If one exists, that table's columns are returned.

[ @pktable_qualifier =] 'pktable_qualifier'

Is the name of the table (with the primary key) qualifier. pktable_qualifier is sysname, with a default of NULL. Various DBMS products support three-part naming for tables (qualifier.owner.name). The qualifier represents the database name. In some products, it represents the server name of the table's database environment.

[ @fktable_name=] 'fktable_name'

Is the name of the table (with a foreign key) used to return catalog information. fktable_name is sysname, with a default of NULL. Wildcard pattern matching is not supported. This parameter or the pktable_name parameter, or both, must be supplied.

[ @fktable_owner =] 'fktable_owner'

Is the name of the owner of the table (with a foreign key) used to return catalog information. fktable_owner is sysname, with a default of NULL. Wildcard pattern matching is not supported. If fktable_owner is not specified, the default table visibility rules of the underlying DBMS apply.

If the current user owns a table with the specified name, that table's columns are returned. If fktable_owner is not specified and the current user does not own a table with the specified fktable_name, the procedure looks for a table with the specified fktable_name owned by the database owner. If one exists, that table's columns are returned.

[ @fktable_qualifier= ] 'fktable_qualifier'

Is the name of the table (with a foreign key) qualifier. fktable_qualifier is sysname, with a default of NULL. The qualifier represents the database name. In some products, it represents the server name of the table's database environment.

Return Code Values

None

Result Sets

Column name | Data type | Description
PKTABLE_QUALIFIER | sysname | Name of the table (with the primary key) qualifier. This field can be NULL.
PKTABLE_OWNER | sysname | Name of the table (with the primary key) owner. This field always returns a value.
PKTABLE_NAME | sysname | Name of the table (with the primary key). This field always returns a value.
PKCOLUMN_NAME | sysname | Name of the primary key columns, for each column of the TABLE_NAME returned. This field always returns a value.
FKTABLE_QUALIFIER | sysname | Name of the table (with a foreign key) qualifier. This field can be NULL.
FKTABLE_OWNER | sysname | Name of the table (with a foreign key) owner. This field always returns a value.
FKTABLE_NAME | sysname | Name of the table (with a foreign key). This field always returns a value.
FKCOLUMN_NAME | sysname | Name of the foreign key column, for each column of the TABLE_NAME returned. This field always returns a value.
KEY_SEQ | smallint | Sequence number of the column in a multicolumn primary key. This field always returns a value.
UPDATE_RULE | smallint | Action applied to the foreign key when the SQL operation is an update. SQL Server returns 0 or 1 for these columns: 0=CASCADE changes to foreign key. 1=NO ACTION changes if foreign key is present.
DELETE_RULE | smallint | Action applied to the foreign key when the SQL operation is a deletion. SQL Server returns 0 or 1 for these columns: 0=CASCADE changes to foreign key. 1=NO ACTION changes if foreign key is present.
FK_NAME | sysname | Foreign key identifier. It is NULL if not applicable to the data source. SQL Server returns the FOREIGN KEY constraint name.
PK_NAME | sysname | Primary key identifier. It is NULL if not applicable to the data source. SQL Server returns the PRIMARY KEY constraint name.

The results returned are ordered byFKTABLE_QUALIFIER, FKTABLE_OWNER, FKTABLE_NAME, and KEY_SEQ.

Remarks

Application coding that includes tables with disabled foreign keys can be implemented by the following:

  • Temporarily disabling constraint checking (ALTER TABLE NOCHECK or CREATE TABLE NOT FOR REPLICATION) while working with the tables, and then enabling it again later.

  • Using triggers or application code to enforce relationships.

If the primary key table name is supplied and the foreign key table name is NULL, sp_fkeys returns all tables that include a foreign key to the given table. If the foreign key table name is supplied and the primary key table name is NULL, sp_fkeys returns all tables related by a primary key/foreign key relationship to foreign keys in the foreign key table.

The sp_fkeys stored procedure is equivalent to SQLForeignKeys in ODBC.

Permissions

Requires SELECT permission on the schema.

Examples

The following example retrieves a list of foreign keys for the DimDate table in the AdventureWorksPDW2012 database. No rows are returned because SQL Server PDW does not support foreign keys.

Copy image Copy Code
EXEC sp_fkeys @pktable_name = N'DimDate;

See Also

Reference

Procedures (SQL Server PDW)
sp_pkeys (SQL Server PDW)


sp_pdw_add_network_credentials (SQL Server PDW)

[image]

sp_pdw_add_network_credentials (SQL Server PDW)
See Also Send Feedback

This stores network credentials in SQL Server PDW and associates them with a server. For example, use this stored procedure to give SQL Server PDW appropriate read/write permissions to perform database backup and restore operations on a target server, or to create a backup of a certificate used for TDE.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_pdw_add_network_credentials 'target_server_name',  'user_name', ꞌpasswordꞌ

Arguments

'target_server_name'

Specifies the target server host name or IP address. SQL Server PDW will access this server by using the username and password credentials passed to this stored procedure.

To connect through the InfiniBand network, use the InfiniBand IP address of the target server.

target_server_name is defined as nvarchar(337).

'user_name'

Specifies the user_name that has permissions to access the target server. If credentials already exist for the target server, they will be updated to the new credentials.

user_name is defined as nvarchar (513).

'passwordꞌ

Specifies the password for user_name.

Return Code Values

0 (success) or 1 (failure)

Permissions

Requires ALTER SERVER STATE permission.

Error Handling

An error occurs if adding credentials does not succeed on the Control node and all Compute nodes.

General Remarks

This stored procedure adds network credentials to the NetworkService account for SQL Server PDW. The NetworkService account runs each instance of SMP SQL Server on the Control node and the Compute nodes. For example, when a backup operation runs, the Control node and each Compute node will use the NetworkService account credentials to gain read and write permission to the target server.

Examples

A. Add credentials for performing a database backup

The following example associates the user name and password credentials for the domain user seattle\david with a target server that has an IP address of 10.172.63.255. The user seattle\david has read/write permissions to the target server. SQL Server PDW will store these credentials and use them to read and write to and from the target server, as necessary for backup and restore operations.

Copy image Copy Code
EXEC sp_pdw_add_network_credentials '10.172.63.255', 'seattle\david', '********';

The backup command requires that the server name be entered as an IP address.

Note Note

To perform the database backup over InfiniBand, be sure to use the InfiniBand IP address of the backup server.

See Also

Reference

BACKUP DATABASE (SQL Server PDW)
sp_pdw_remove_network_credentials (SQL Server PDW)


sp_pdw_database_encryption (SQL Server PDW)

[image]

sp_pdw_database_encryption (SQL Server PDW)
Example See Also Send Feedback

Use sp_pdw_database_encryption to enable transparent data encryption on for a SQL Server PDW appliance. When sp_pdw_database_encryption set to 1, use the ALTER DATABASE statement to encrypt a database by using TDE.

Syntax

Transact-SQL
sp_pdw_database_encryption [ [ @enabled = ] enabled ] ;

Parameters

[ @enabled= ] enabled

Determines whether transparent data encryption is enabled. enabled is int, and can be one of the following values:

  • 0 = Disabled

  • 1 = Enabled

Executing sp_pdw_database_encryption without parameters returns the current state of TDE on the appliance as a scalar result set: 0 for disabled, or 1 for enabled.

Return Code Values

0 (success) or 1 (failure)

Remarks

When the TDE is enabled using sp_pdw_database_encryption, the tempdb database is dropped, recreated and encrypted. For that reason, the TDE cannot be enabled on an appliance while there are other active sessions using tempdb. Enabling or disabling TDE on an appliance is an action that changes the state of the appliance, in most cases is expected to be performed once in the appliance lifetime, and should be executed when there is no traffic on the appliance.

Permissions

Requires membership in the sysadmin fixed database role, or CONTROL SERVER permission.

Example

The following example enables TDE on the appliance.

Transact-SQL Copy image Copy Code
EXEC sys.sp_pdw_database_encryption 1;

See Also

Reference

sp_pdw_database_encryption_regenerate_system_keys (SQL Server PDW)
sp_pdw_log_user_data_masking (SQL Server PDW)


sp_pdw_database_encryption_regenerate_system_keys (SQL Server PDW)

[image]

sp_pdw_database_encryption_regenerate_system_keys (SQL Server PDW)
Example See Also Send Feedback

Use sp_pdw_database_encryption_regenerate_system_keys to rotate the certificate and database encryption key for internal databases that are encrypted when TDE is enabled on the appliance. This includes tempdb. This will succeed only if TDE is enabled.

Syntax

Transact-SQL
sp_pdw_database_encryption_regenerate_system_keys  ;

Return Code Values

0 (success) or 1 (failure)

Remarks

The procedure has no parameters.

This procedure should be used when the traffic in the appliance is low.

Permissions

Requires membership in the sysadmin fixed database role, or CONTROL SERVER permission.

Example

The following example regenerates the database encryption keys.

Transact-SQL Copy image Copy Code
EXEC sys.sp_pdw_database_encryption_regenerate_system_keys;

See Also

Reference

sp_pdw_database_encryption (SQL Server PDW)
sp_pdw_log_user_data_masking (SQL Server PDW)


sp_pdw_log_user_data_masking (SQL Server PDW)

[image]

sp_pdw_log_user_data_masking (SQL Server PDW)
Example See Also Send Feedback

Use sp_pdw_log_user_data_masking to enable user data masking in PDW activity logs. User data masking affects the statements on all databases on the appliance.

Important note Important

The PDW activity logs affected by sp_pdw_log_user_data_masking are certain PDW activity logs. sp_pdw_log_user_data_masking does not affect database transaction logs, or SQL Server error logs.

Background: In the default configuration PDW activity logs contain full Transact-SQL statements, and can in some cases include user data contained in operations such as INSERT, UPDATE, and SELECT statements. In case of a problem on the appliance, this permits the analysis of the conditions that caused the problem without a need to reproduce the issue. In order to prevent the user data from being written to PDW activity logs, customers can choose to turn on the user data masking by using this stored procedure. The statements will still be written to PDW activity logs, but all the literals in statements that may contain user data will be masked; replaced with some predefined constant values.

When transparent data encryption is enabled on the appliance, masking of the user data in PDW activity logs is automatically turned on. For more information about TDE, see Transparent Data Encryption (SQL Server PDW).

Syntax

sp_pdw_log_user_data_masking [ [ @masking_mode = ] value ] ;

Parameters

[ @masking_mode= ] masking_mode

Determines whether transparent data encryption log user data masking is enabled. masking_mode is int, and can be one of the following values:

  • 0 = Disabled, user data appears in the PDW activity logs.

  • 1 = Enabled, user data statements appear in the PDW activity logs but the user data is masked.

  • 2 = Statements containing user data are not written to the PDW activity logs.

Executing sp_pdw_ log_user_data_masking without parameters returns the current state of TDE log user data masking on the appliance as a scalar result set.

Remarks

User data masking in PDW activity logs enables replacement of literals with predefined constant values in SELECT and DML statements, as they can contain user data. Setting masking_mode to 1 does not mask metadata, such as column names or table names. Setting masking_mode to 2 removes statements with metadata, such as column names or table names.

User data masking in PDW activity logs is implemented in the following way:

  • TDE and user data masking in PDW activity logs are turned off by default. The statements will not be automatically masked if database encryption is not enabled on the appliance.

  • Enabling TDE on the appliance automatically turns on the user data masking in PDW activity logs.

  • Disabling TDE does not affect user data masking in PDW activity logs.

  • You can explicitly enable user data masking in PDW activity logs by using the sp_pdw_log_user_data_masking procedure.

Permissions

Requires membership in the sysadmin fixed database role, or CONTROL SERVER permission.

Example

The following example enables TDE log user data masking on the appliance.

Copy image Copy Code
EXEC sp_pdw_log_user_data_masking 1;

See Also

Reference

sp_pdw_database_encryption (SQL Server PDW)
sp_pdw_database_encryption_regenerate_system_keys (SQL Server PDW)


sp_pdw_remove_network_credentials (SQL Server PDW)

[image]

sp_pdw_remove_network_credentials (SQL Server PDW)
See Also Send Feedback

This removes network credentials stored in SQL Server PDW to access a network file share. For example, use this stored procedure to remove permission for SQL Server PDW to perform backup and restore operations on a server that resides within your own network.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_pdw_remove_network_credentials 'target_server_name'

Arguments

'target_server_name'

Specifies the target server host name or IP address. Credentials to access this server will be removed from SQL Server PDW. This does not change or remove any permissions on the actual target server which is managed by your own team.

target_server_name is defined as nvarchar(337).

Return Code Values

0 (success) or 1 (failure)

Permissions

Requires ALTER SERVER STATE permission.

Error Handling

An error occurs if removing credentials does not succeed on the Control node and all Compute nodes.

General Remarks

This stored procedure removes network credentials from the NetworkService account for SQL Server PDW. The NetworkService account runs each instance of SMP SQL Server on the Control node and the Compute nodes. For example, when a backup operation runs, the Control node and each Compute node will use the NetworkService account credentials to access the target server.

Metadata

To list all credentials and to verify the credentials have been removed, use sys.dm_pdw_network_credentials (SQL Server PDW).

To add credentials, use sp_pdw_add_network_credentials (SQL Server PDW).

Examples

A. Remove credentials for performing a database backup

The following example removes user name and password credentials for accessing the target server which has an IP address of 10.192.147.63.

Copy image Copy Code
EXEC sp_pdw_remove_network_credentials '10.192.147.63';

See Also

Reference

BACKUP DATABASE (SQL Server PDW)


sp_pkeys (SQL Server PDW)

[image]

sp_pkeys (SQL Server PDW)
See Also Send Feedback

Returns primary key information for a single table in the current environment.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_pkeys [ @table_name = ] 'name'
      [ , [ @table_owner = ] 'owner' ]
   [ , [ @table_qualifier = ] 'qualifier' ]

Arguments

[ @table_name= ] 'name'

Is the table for which to return information. name is sysname, with no default. Wildcard pattern matching is not supported.

[ @table_owner= ] 'owner'

Specifies the table owner of the specified table. owner is sysname, with a default of NULL. Wildcard pattern matching is not supported. If owner is not specified, the default table visibility rules of the underlying DBMS apply.

In SQL Server, if the current user owns a table with the specified name, the columns of that table are returned. If the owner is not specified and the current user does not own a table with the specified name, this procedure looks for a table with the specified name owned by the database owner. If one exists, the columns of that table are returned.

[ @table_qualifier= ] 'qualifier'

Is the table qualifier. qualifier is sysname, with a default of NULL. Various DBMS products support three-part naming for tables (qualifier.owner.name). In SQL Server, this column represents the database name. In some products, it represents the server name of the database environment of the table.

Return Code Values

None

Result Sets

Column name | Data type | Description
TABLE_QUALIFIER | sysname | Name of the table qualifier. This field can be NULL.
TABLE_OWNER | sysname | Name of the table owner. This field always returns a value.
TABLE_NAME | sysname | Name of the table. In SQL Server, this column represents the table name as listed in the sysobjects table. This field always returns a value.
COLUMN_NAME | sysname | Name of the column, for each column of the TABLE_NAME returned. In SQL Server, this column represents the column name as listed in the sys.columns table. This field always returns a value.
KEY_SEQ | smallint | Sequence number of the column in a multicolumn primary key.
PK_NAME | sysname | Primary key identifier. Returns NULL if not applicable to the data source.

Remarks

sp_pkeys returns information about columns explicitly defined with a PRIMARY KEY constraint. Because not all systems support explicitly named primary keys, the gateway implementer determines what constitutes a primary key. Note that the term primary key refers to a logical primary key for a table. It is expected that every key listed as being a logical primary key has a unique index defined on it. This unique index is also returned in sp_statistics.

The sp_pkeys stored procedure is equivalent to SQLPrimaryKeys in ODBC. The results returned are ordered by TABLE_QUALIFIER, TABLE_OWNER, TABLE_NAME, and KEY_SEQ.

Permissions

Requires SELECT permission on the schema.

Examples

The following example retrieves the primary key for the DimAccount table in the AdventureWorksPDW2012 database. It returns zero rows indicating that the table does not have a primary key.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
EXEC sp_pkeys @table_name = N'DimAccount;

See Also

Reference

Procedures (SQL Server PDW)
sp_fkeys (SQL Server PDW)


sp_prepare (SQL Server PDW)

[image]

sp_prepare (SQL Server PDW)
See Also Send Feedback

Prepares a parameterized Transact-SQL statement and returns a statement handle for execution. sp_prepare is invoked by specifying ID = 11 in a tabular data stream (TDS) packet.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_prepare handle OUTPUT, params, stmt, options

Arguments

handle

Is a SQL Server-generated prepared handle identifier. handle is a required parameter with an int return value.

params

Identifies parameterized statements. The params definition of variables is substituted for parameter markers in the statement. params is a required parameter that calls for an ntext, nchar, or nvarchar input value. Input a NULL value if the statement is not parameterized.

stmt

Defines the batch statement. The stmt parameter is required and calls for an ntext, nchar, or nvarchar input value.

options

An optional parameter that returns a description of the result set columns. options requires the following int input value.

Value

Description

0x0001

RETURN_METADATA

Note Note

Currently metadata does not display when using a question mark in the projection list.

Examples

The following example prepares and executes a simple statement.

Copy image Copy Code
Declare @P1 int;
Exec sp_prepare @P1 output,
    N'@P1 nvarchar(128), @P2 nvarchar(100)',
    N'SELECT database_id, name FROM sys.databases WHERE name=@P1 AND state_desc = @P2';
Exec sp_execute @P1, N'tempdb', N'ONLINE';
EXEC sp_unprepare @P1;

See Also

Reference

Procedures (SQL Server PDW)
sp_unprepare (SQL Server PDW)
sp_execute (SQL Server PDW)


sp_special_columns_100 (SQL Server PDW)

[image]

sp_special_columns_100 (SQL Server PDW)
See Also Send Feedback

Returns the optimal set of columns that uniquely identify a row in the table. Also returns columns automatically updated when any value in the row is updated by a transaction.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_special_columns_100 [ @table_name = ] 'table_name'
     [ , [ @table_owner = ] 'table_owner' ]
     [ , [ @qualifier = ] 'qualifier' ]
     [ , [ @col_type = ] 'col_type' ]
     [ , [ @scope = ] 'scope' ]
     [ , [ @nullable = ] 'nullable' ]
     [ , [ @ODBCVer = ] 'ODBCVer' ]
[ ; ]

Arguments

[ @table_name=] 'table_name'

Is the name of the table used to return catalog information. name is sysname, with no default. Wildcard pattern matching is not supported.

[ @table_owner=] 'table_owner'

Is the table owner of the table used to return catalog information. owner is sysname, with a default of NULL. Wildcard pattern matching is not supported. If owner is not specified, the default table visibility rules of the underlying DBMS apply.

In SQL Server, if the current user owns a table with the specified name, the columns of that table are returned. If owner is not specified and the current user does not own a table of the specified name, this procedure looks for a table of the specified name owned by the database owner. If the table exists, its columns are returned.

[ @qualifier=] 'qualifier'

Is the name of the table qualifier. qualifier is sysname, with a default of NULL. Various DBMS products support three-part naming for tables (qualifier.owner.name). In SQL Server, this column represents the database name. In some products, it represents the server name of the database environment of the table.

[ @col_type=] 'col_type'

Is the column type. col_type is char(1), with a default of R. Type R returns the optimal column or set of columns that, by retrieving values from the column or columns, allows for any row in the specified table to be uniquely identified. A column can be either a pseudocolumn specifically designed for this purpose, or the column or columns of any unique index for the table. Type V returns the column or columns in the specified table, if any, that are automatically updated by the data source when any value in the row is updated by any transaction.

[ @scope=] 'scope'

Is the minimum required scope of the ROWID. scope is char(1), with a default of T. Scope C specifies that the ROWID is valid only when positioned on that row. Scope T specifies that the ROWID is valid for the transaction.

[ @nullable=] 'nullable'

Is whether the special columns can accept a null value. nullable is char(1), with a default of U. O specifies special columns that do not allow null values. U specifies columns that are partially nullable.

[ @ODBCVer=] 'ODBCVer'

Is the ODBC version being used. ODBCVer is int(4), with a default of 2. This indicates ODBC version 2.0. For more information about the difference between ODBC version 2.0 and ODBC version 3.0, see the ODBC SQLSpecialColumns specification for ODBC version 3.0.

Return Code Values

None

Result Sets

Column name

Data type

Description

SCOPE

smallint

Actual scope of the row ID. Can be 0, 1, or 2. SQL Server always returns 0. This field always returns a value.

0 = SQL_SCOPE_CURROW. The row ID is guaranteed to be valid only while positioned on that row. A later reselect using the row ID may not return a row if the row was updated or deleted by another transaction.

1 = SQL_SCOPE_TRANSACTION. The row ID is guaranteed to be valid for the duration of the current transaction.

2 = SQL_SCOPE_SESSION. The row ID is guaranteed to be valid for the duration of the session (across transaction boundaries).

COLUMN_NAME

sysname

Column name for each column of the tablereturned. This field always returns a value.

DATA_TYPE

smallint

ODBC SQL data type.

TYPE_NAME

sysname

Data source-dependent data type name; for example, char, varchar, money, or text.

PRECISION

Int

Precision of the column on the data source. This field always returns a value.

LENGTH

Int

Length, in bytes, required for the data type in its binary form in the data source, for example, 10 for char(10), 4 for integer, and 2 for smallint.

SCALE

smallint

Scale of the column on the data source. NULL is returned for data types for which scale is not applicable.

PSEUDO_COLUMN

smallint

Indicates whether the column is a pseudocolumn. SQL Server always returns 1:

0 = SQL_PC_UNKNOWN

1 = SQL_PC_NOT_PSEUDO

2 = SQL_PC_PSEUDO

Remarks

sp_special_columns is equivalent to SQLSpecialColumns in ODBC. The results returned are ordered by SCOPE.

Permissions

Requires SELECT permission on the schema.

Examples

The following example returns information about the column that uniquely identifies rows in the FactFinance table.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
EXEC sp_special_columns_100 @table_name = 'FactFinance';

See Also

Reference

Procedures (SQL Server PDW)


sp_sproc_columns (SQL Server PDW)

[image]

sp_sproc_columns (SQL Server PDW)
See Also Send Feedback

Returns column information for a single stored procedure or user-defined function in the current database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_sproc_columns [[@procedure_name = ] 'name']
    [ , [@procedure_owner = ] 'owner']
    [ , [@procedure_qualifier = ] 'qualifier']
    [ , [@column_name = ] 'column_name']
    [ , [@ODBCVer = ] 'ODBCVer']
    [ , [@fUsePattern = ] 'fUsePattern']

Arguments

[ @procedure_name = ] 'name'

Is the name of the procedure used to return catalog information. name is nvarchar(390), with a default of %, which means all tables in the current database. Wildcard pattern matching is supported.

[ @procedure_owner =] 'owner'

Is the name of the owner of the procedure. owneris nvarchar(384), with a default of NULL. Wildcard pattern matching is supported. If owner is not specified, the default procedure visibility rules of the underlying DBMS apply.

If the current user owns a procedure with the specified name, information about that procedure is returned. If owneris not specified and the current user does not own a procedure with the specified name, sp_sproc_columns looks for a procedure with the specified name that is owned by the database owner. If the procedure exists, information about its columns is returned.

[ @procedure_qualifier =] 'qualifier'

Is the name of the procedure qualifier. qualifier is sysname, with a default of NULL. Various DBMS products support three-part naming for tables (qualifier.owner.name). In SQL Server, this parameter represents the database name. In some products, it represents the server name of the table's database environment.

[ @column_name =] 'column_name'

Is a single column and is used when only one column of catalog information is desired. column_name is nvarchar(384), with a default of NULL. If column_name is omitted, all columns are returned. Wildcard pattern matching is supported. For maximum interoperability, the gateway client should assume only ISO standard pattern matching (the % and _ wildcard characters).

[ @ODBCVer =] 'ODBCVer'

Is the version of ODBC being used. ODBCVer is int, with a default of 2, which indicates ODBC version 2.0. For more information about the difference between ODBC version 2.0 and ODBC version 3.0, refer to the ODBC SQLProcedureColumns specification for ODBC version 3.0

[ @fUsePattern =] 'fUsePattern'

Determines whether the underscore (_), percent (%), and bracket ([ ]) characters are interpreted as wildcard characters. Valid values are 0 (pattern matching is off) and 1 (pattern matching is on). fUsePattern is bit, with a default of 1.

Return Code Values

None

Result Sets

Column name | Data type | Description
PROCEDURE_QUALIFIER | sysname | Procedure qualifier name. This column can be NULL.
PROCEDURE_OWNER | sysname | Procedure owner name. This column always returns a value.
PROCEDURE_NAME | nvarchar(134) | Procedure name. This column always returns a value.
COLUMN_NAME | sysname | Column name for each column of the TABLE_NAME returned. This column always returns a value.
COLUMN_TYPE | smallint | This field always returns a value: 0 = SQL_PARAM_TYPE_UNKNOWN 1 = SQL_PARAM_TYPE_INPUT 2 = SQL_PARAM_TYPE_OUTPUT 3 = SQL_RESULT_COL 4 = SQL_PARAM_OUTPUT 5 = SQL_RETURN_VALUE
DATA_TYPE | smallint | Integer code for an ODBC data type. If this data type cannot be mapped to an ISO type, the value is NULL. The native data type name is returned in the TYPE_NAME column.
TYPE_NAME | sysname | String representation of the data type. This is the data type name as presented by the underlying DBMS.
PRECISION | int | Number of significant digits. The return value for the PRECISION column is in base 10.
LENGTH | int | Transfer size of the data.
SCALE | smallint | Number of digits to the right of the decimal point.
RADIX | smallint | Is the base for numeric types.
NULLABLE | smallint | Specifies nullability: 1 = Data type can be created allowing null values. 0 = Null values are not allowed.
REMARKS | varchar(254) | Description of the procedure column. SQL Server does not return a value for this column.
COLUMN_DEF | nvarchar(4000) | Default value of the column.
SQL_DATA_TYPE | smallint | Value of the SQL data type as it appears in the TYPE field of the descriptor. This column is the same as the DATA_TYPE column, except for the datetime and ISO interval data types. This column always returns a value.
SQL_DATETIME_SUB | smallint | The datetime ISO interval subcode if the value of SQL_DATA_TYPE is SQL_DATETIME or SQL_INTERVAL. For data types other than datetime and ISO interval, this field is NULL.
CHAR_OCTET_LENGTH | int | Maximum length in bytes of a character or binary data type column. For all other data types, this column returns a NULL.
ORDINAL_POSITION | int | Ordinal position of the column in the table. The first column in the table is 1. This column always returns a value.
IS_NULLABLE | varchar(254) | Nullability of the column in the table. ISO rules are followed to determine nullability. An ISO compliant DBMS cannot return an empty string. Displays YES if the column can include NULLS and NO if the column cannot include NULLS. This column returns a zero-length string if nullability is unknown. The value returned for this column is different from the value returned for the NULLABLE column.
SS_DATA_TYPE | tinyint | SQL Server data type used by extended stored procedures. For more information, see Data Types (SQL Server PDW).

Remarks

sp_sproc_columns is equivalent to SQLProcedureColumns in ODBC. The results returned are ordered by PROCEDURE_QUALIFIER, PROCEDURE_OWNER, PROCEDURE_NAME, and the order that the parameters appear in the procedure definition.

Permissions

Requires SELECT permission on the schema of the stored procedure.

See Also

Reference

Procedures (SQL Server PDW)


sp_statistics (SQL Server PDW)

[image]

sp_statistics (SQL Server PDW)
See Also Send Feedback

Returns a list of all indexes and statistics on a specified table or indexed view.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_statistics [ @table_name = ] 'table_name'
     [ , [ @table_owner = ] 'owner' ]
     [ , [ @table_qualifier = ] 'qualifier' ]
     [ , [ @index_name = ] 'index_name' ]
     [ , [ @is_unique = ] 'is_unique' ]
     [ , [ @accuracy = ] 'accuracy' ]

Arguments

[ @table_name= ] 'table_name'

Specifies the table used to return catalog information. table_name is sysname, with no default. Wildcard pattern matching is not supported.

[ @table_owner= ] 'owner'

Is the name of the table owner of the table used to return catalog information. table_owner is sysname, with a default of NULL. Wildcard pattern matching is not supported. If owner is not specified, the default table visibility rules of the underlying DBMS apply.

In SQL Server, if the current user owns a table with the specified name, the indexes of that table are returned. If owner is not specified and the current user does not own a table with the specified name, this procedure looks for a table with the specified name owned by the database owner. If one exists, the indexes of that table are returned.

[ @table_qualifier= ] 'qualifier'

Is the name of the table qualifier. qualifier is sysname, with a default of NULL. Various DBMS products support three-part naming for tables (qualifier.owner.name). In SQL Server, this parameter represents the database name. In some products, it represents the server name of the table's database environment.

[ @index_name= ] 'index_name'

Is the index name. index_name is sysname, with a default of %. Wildcard pattern matching is supported.

[ @is_unique= ] 'is_unique'

Is whether only unique indexes (if Y) are to be returned. is_unique is char(1), with a default of N.

[ @accuracy= ] 'accuracy'

Is the level of cardinality and page accuracy for statistics. accuracy is char(1), with a default of Q. Specify E to make sure that statistics are updated so that cardinality and pages are accurate.

The value E (SQL_ENSURE) asks the driver to unconditionally retrieve the statistics.

The value Q (SQL_QUICK) asks the driver to retrieve the cardinality and pages only if they are readily available from the server. In this case, the driver does not ensure that the values are current. Applications that are written to the Open Group standard will always get SQL_QUICK behavior from ODBC 3.x-compliant drivers.

Result Sets

Column name | Data type | Description
TABLE_QUALIFIER | sysname | Table qualifier name. This column can be NULL.
TABLE_OWNER | sysname | Table owner name. This column always returns a value.
TABLE_NAME | sysname | Table name. This column always returns a value.
NON_UNIQUE | smallint | NOT NULL. 0 = Unique 1 = Not unique
INDEX_QUALIFIER | sysname | Index owner name. Some DBMS products allow for users other than the table owner to create indexes. In SQL Server, this column is always the same as TABLE_NAME.
INDEX_NAME | sysname | Is the name of the index. This column always returns a value.
TYPE | smallint | This column always returns a value: 0 = Statistics for a table 1 = Clustered 2 = Hashed 3 = Nonclustered
SEQ_IN_INDEX | smallint | Position of the column within the index.
COLUMN_NAME | sysname | Column name for each column of the TABLE_NAME returned. This column always returns a value.
COLLATION | char(1) | Order used in collation. Can be: A = Ascending D = Descending NULL = Not applicable
CARDINALITY | int | Number of rows in the table or unique values in the index.
PAGES | int | Number of pages to store the index or table.
FILTER_CONDITION | varchar(128) | SQL Server does not return a value.

Return Code Values

None

Remarks

The indexes in the result set appear in ascending order by the columns NON_UNIQUE, TYPE, INDEX_NAME, and SEQ_IN_INDEX.

The index type clustered refers to an index in which table data is stored in the order of the index. This corresponds to SQL Server clustered indexes.

The index type Hashed accepts exact match or range searches, but pattern matching searches do not use the index.

sp_statistics is equivalent to SQLStatistics in ODBC. The results returned are ordered by NON_UNIQUE, TYPE, INDEX_QUALIFIER, INDEX_NAME, and SEQ_IN_INDEX. For more information, see the ODBC API Reference.

Permissions

Requires SELECT permission on the schema.

Example

The following example returns information about the DimEmployee table.

Copy image Copy Code
USE AdventureWorksPDW2012;
EXEC sp_statistics DimEmployee;

See Also

Reference

Procedures (SQL Server PDW)


sp_tables (SQL Server PDW)

[image]

sp_tables (SQL Server PDW)
See Also Send Feedback

Returns a list of objects that can be queried in the current environment. This means any table or view, except synonym objects.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_tables [ [ @table_name = ] 'name' ]
     [ , [ @table_owner = ] 'owner' ]
     [ , [ @table_qualifier = ] 'qualifier' ]
     [ , [ @table_type = ] "type" ]
     [ , [@fUsePattern = ] 'fUsePattern'];

Arguments

[ @table_name= ] 'name'

Is the table used to return catalog information. name is nvarchar(384), with a default of NULL. Wildcard pattern matching is supported.

[ @table_owner= ] 'owner'

Is the table owner of the table used to return catalog information. owner is nvarchar(384), with a default of NULL. Wildcard pattern matching is supported. If the owner is not specified, the default table visibility rules of the underlying DBMS apply.

In SQL Server, if the current user owns a table with the specified name, the columns of that table are returned. If the owner is not specified and the current user does not own a table with the specified name, this procedure looks for a table with the specified name owned by the database owner. If one exists, the columns of that table are returned.

[ @table_qualifier= ] 'qualifier'

Is the name of the table qualifier. qualifier is sysname, with a default of NULL. Various DBMS products support three-part naming for tables (qualifier.owner.name). In SQL Server, this column represents the database name. In some products, it represents the server name of the table's database environment.

[ , [ @table_type= ] "'type', 'type'" ]

Is a list of values, separated by commas, that gives information about all tables of the table types that are specified. These include TABLE, SYSTEMTABLE, and VIEW. type is varchar(100), with a default of NULL.

Note Note

Single quotation marks must enclose each table type, and double quotation marks must enclose the whole parameter. Table types must be uppercase. If SET QUOTED_IDENTIFIER is ON, each single quotation mark must be doubled and the whole parameter must be enclosed in single quotation marks.

[ @fUsePattern = ] 'fUsePattern'

Determines whether the underscore ( _ ), percent ( % ), and bracket ( [ or ] ) characters are interpreted as wildcard characters. Valid values are 0 (pattern matching is off) and 1 (pattern matching is on). fUsePattern is bit, with a default of 1.

Return Code Values

None

Result Sets

Column name | Data type | Description
TABLE_QUALIFIER | sysname | Table qualifier name. In SQL Server, this column represents the database name. This field can be NULL.
TABLE_OWNER | sysname | Table owner name. In SQL Server, this column represents the name of the database user who created the table. This field always returns a value.
TABLE_NAME | sysname | Table name. This field always returns a value.
TABLE_TYPE | varchar(32) | Table, system table, or view.
REMARKS | varchar(254) | SQL Server does not return a value for this column.

Remarks

For maximum interoperability, the client should assume only SQL-92-standard SQL pattern matching (the % and _ wildcard characters).

sp_tables is equivalent to SQLTables in ODBC. The results returned are ordered by TABLE_TYPE, TABLE_QUALIFIER, TABLE_OWNER, and TABLE_NAME.

Permissions

Requires SELECT permission on the schema.

Examples

A. Returning a list of objects that can be queried in the current environment

The following example returns a list of objects that can be queries in the current environment.

Copy image Copy Code
EXEC sp_tables ;

B. Returning information about the tables in a specified schema

The following example returns information about the dimension tables in the AdventureWorksPDW201 database.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
EXEC sp_tables
   @table_name = 'Dim%',
   @table_owner = 'dbo',
   @table_qualifier = 'AdventureWorksPDW2012';

See Also

Reference

Procedures (SQL Server PDW)


sp_unprepare (SQL Server PDW)

[image]

sp_unprepare (SQL Server PDW)
See Also Send Feedback

Discards the execution plan created by the sp_prepare stored procedure. sp_unprepare is invoked by specifying ID = 15 in a tabular data stream (TDS) packet.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

sp_unprepare handle

Arguments

handle

Is the handle value returned by sp_prepare.

Examples

The following example prepares, executes, and unprepares a simple statement.

Copy image Copy Code
Declare @P1 int;
Exec sp_prepare @P1 output,
    N'@P1 nvarchar(128), @P2 nvarchar(100)',
    N'SELECT database_id, name FROM sys.databases WHERE name=@P1 AND state_desc = @P2';
Exec sp_execute @P1, N'tempdb', N'ONLINE';
EXEC sp_unprepare @P1;

See Also

Reference

Procedures (SQL Server PDW)
sp_prepare (SQL Server PDW)
sp_execute (SQL Server PDW)


SET Statements (SQL Server PDW)

[image]

SET Statements (SQL Server PDW)
See Also Send Feedback

SQL Server PDW supports several SET statements that affect the way the current session handles specific information. There is no requirement to use the SET statements; they are provided for compatibility with client tools and existing scripts.

SET Statements

The SET statements are based on the SQL Server SET statements. However, most SET statements in SQL Server PDW have only one valid value, which is already set. Therefore, there is no requirement to use the SET statements. As mentioned in the introduction, they are provided for compatibility with client tools and existing scripts.

This SQL Server PDW documentation contains the SET statements supported by SQL Server PDW. For a full explanation of each SET statement, see the SQL Server documentation. The documentation links and valid values are described in the following table.

Supported Setting | Valid Value | SQL Server PDW Documentation | SQL Server Documentation
SET ANSI_DEFAULTS | ON | SET ANSI_DEFAULTS (SQL Server PDW) | SET ANSI_DEFAULTS (Transact-SQL)
SET ANSI_NULL_DFLT_OFF | OFF | SET ANSI_NULL_DFLT_OFF (SQL Server PDW) | SET ANSI_NULL_DFLT_OFF (Transact-SQL)
SET ANSI_NULL_DFLT_ON | ON | SET ANSI_NULL_DFLT_ON (SQL Server PDW) | SET ANSI_NULL_DFLT_ON (Transact-SQL
SET ANSI_NULLS | ON | SET ANSI_NULLS (SQL Server PDW) | SET ANSI_NULLS (Transact-SQL)
SET ANSI_PADDING | ON | SET ANSI_PADDING (SQL Server PDW) | SET ANSI_PADDING (Transact-SQL)
SET ANSI_WARNINGS | ON | SET ANSI_WARNINGS (SQL Server PDW) | SET ANSI_WARNINGS (Transact-SQL)
SET ARITHABORT | ON | SET ARITHABORT (SQL Server PDW) | SET ARITHABORT (Transact-SQL)
SET ARITHIGNORE | OFF | SET ARITHIGNORE (SQL Server PDW) | SET ARITHIGNORE (Transact-SQL)
SET CONCAT_NULL_YIELDS_NULL | ON | SET CONCAT_NULL_YIELDS_NULL (SQL Server PDW) | SET CONCAT_NULL_YIELDS_NULL (Transact-SQL)
SET DATEFIRST | 7 | SET DATEFIRST (SQL Server PDW) | SET DATEFIRST (Transact-SQL)
SET DATEFORMAT | mdy | SET DATEFORMAT (SQL Server PDW) | SET DATEFIRST (Transact-SQL)
SET FMTONLY | ON | OFF | SET FMTONLY (SQL Server PDW) | SET FMTONLY (Transact-SQL)
SET IMPLICIT_TRANSACTIONS | ON | OFF | SET IMPLICIT_TRANSACTIONS (SQL Server PDW) | SET IMPLICIT_TRANSACTIONS Transact-SQL
SET LOCK_TIMEOUT | -1 Values > -1 are valid, but ignored. | SET LOCK_TIMEOUT (SQL Server PDW) | SET LOCK_TIMEOUT (Transact-SQL)
SET NUMERIC_ROUNDABORT | ON | SET NUMERIC_ROUNDABORT (SQL Server PDW) | SET NUMERIC_ROUNDABORT (Transact-SQL)
SET QUOTED_IDENTIFIER | ON | SET QUOTED_IDENTIFIER (SQL Server PDW) | SET QUOTED_IDENTIFIER (Transact-SQL)
SET ROWCOUNT | n | SET ROWCOUNT (SQL Server PDW) | SET ROWCOUNT (Transact-SQL)
SET TEXTSIZE | Max is 2147483648. | SET TEXTSIZE (SQL Server PDW) | SET TEXTSIZE (Transact-SQL)
SET TRANSACTION ISOLATION LEVEL | READ UNCOMMITTED | SET TRANSACTION ISOLATION LEVEL (SQL Server PDW) | SET TRANSACTION ISOLATION LEVEL (Transact-SQL)
SET XACT_ABORT | ON | SET XACT_ABORT (SQL Server PDW) | SET XACT_ABORT (Transact-SQL)

See Also


SET ANSI_DEFAULTS (SQL Server PDW)

[image]

SET ANSI_DEFAULTS (SQL Server PDW)
See Also Send Feedback

Controls a group of SQL Server PDW SQL settings that collectively specify some ISO standard behavior.

In this release, ANSI_DEFAULTS is always ON.

For more information, see the SET ANSI_DEFAULTS (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET ANSI_DEFAULTS ON;

Permissions

Requires membership in the public role.

General Remarks

ANSI_DEFAULTS is a server-side setting that the client does not modify.

The SQL Server Native Client ODBC driver and SQL Server Native Client OLE DB Provider for SQL Server automatically set ANSI_DEFAULTS to ON when connecting.

See Also


SET ANSI_NULL_DFLT_OFF (SQL Server PDW)

[image]

SET ANSI_NULL_DFLT_OFF (SQL Server PDW)
See Also Send Feedback

Alters the behavior of the session to override default nullability of new columns when the ANSI_NULL_DFLT_ON setting is ON.

In this release, ANSI_NULL_DFLT_OFF is always OFF and ANSI_NULL_DFLT_ON is always ON. Therefore, new columns are always nullable unless NOT NULL is explicitly stated in the column definition.

For more information, see the SET ANSI_NULL_DFLT_OFF (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET ANSI_NULL_DFLT_OFF OFF

Permissions

Requires membership in the public role.

General Remarks

This setting only affects the nullability of new columns when the nullability of the column is not specified in the CREATE TABLE and ALTER TABLE statements. New columns that are specified with an explicit NOT NULL are not affected by the ANSI_NULL_DFLT_ON setting.

The setting of SET ANSI_NULL_DFLT_OFF is set at execute or run time and not at parse time.

See Also


SET ANSI_NULL_DFLT_ON (SQL Server PDW)

[image]

SET ANSI_NULL_DFLT_ON (SQL Server PDW)
See Also Send Feedback

Alters the behavior of the SQL Server PDW session to override default nullability of new columns when the ANSI_NULL_DFLT_OFF setting is OFF.

In this release, ANSI_NULL_DFLT_ON is always ON and ANSI_NULL_DFLT_OFF is always OFF. Therefore, new columns are always nullable unless NOT NULL is explicitly stated in the column definition.

For more information, see the SET ANSI_NULL_DFLT_ON (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET ANSI_NULL_DFLT_ON ON;

Permissions

Requires membership in the public role.

General Remarks

This setting only affects the nullability of new columns when the nullability of the column is not specified in the CREATE TABLE and ALTER TABLE statements. New columns that are specified with an explicit NOT NULL are not affected by the ANSI_NULL_DFLT_ON setting.

The SET ANSI_NULL_DFLT_OFF is set at execute or run time and not at parse time.

Permissions

Requires membership in the public role.

See Also


SET ANSI_NULLS (SQL Server PDW)

[image]

SET ANSI_NULLS (SQL Server PDW)
See Also Send Feedback

Specifies ISO compliant behavior of the Equals (=) and Not Equal To (<>) comparison operators when they are used with NULL values in SQL Server PDW.

In this release, ANSI_NULLS is always ON.

For more information, see the SET ANSI_NULLS (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET ANSI_NULLS ON;

Permissions

Requires membership in the public role.

General Remarks

When ANSI_NULLS is ON:

  1. A SELECT statement that uses WHERE column_name = NULL returns zero rows even if there are NULL values in column_name. A SELECT statement that uses WHERE column_name <> NULL returns zero rows even if there are nonnull values in column_name.

  2. All comparisons against a NULL value evaluate to UNKNOWN.

  3. The setting affects a comparison only if one of the operands of the comparison is either a variable that is NULL or a literal NULL. If both sides of the comparison are columns or compound expressions, the setting does not affect the comparison.

For a script to work as intended, regardless of the ANSI_NULLS setting, use IS NULL and IS NOT NULL in comparisons that might contain NULL values.

The SQL Server Native Client ODBC driver and SQL Server Native Client OLE DB Provider for SQL Server automatically set ANSI_NULLS to ON when connecting. This setting can be configured in ODBC data sources, in ODBC connection attributes, or in OLE DB connection properties that are set in the application before connecting to an instance of SQL Server PDW.

The setting of SET ANSI_NULLS is set at execute or run time and not at parse time.

See Also


SET ANSI_PADDING (SQL Server PDW)

[image]

SET ANSI_PADDING (SQL Server PDW)
See Also Send Feedback

Specifies the way SQL Server PDW stores column values shorter than the defined size of the column, and stores column values that have trailing spaces in char, varchar, binary, and varbinary data.

In this release, ANSI_PADDING is always ON.

For more information, see the SET ANSI_PADDING (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET ANSI_PADDING ON;

Permissions

Requires membership in the public role.

General Remarks

When ANSI_PADDING is ON, and a column value is inserted into a column that is shorter than the defined size:

  • char (n) values are padded with trailing spaces.

  • binary (n) values are padded with trailing zeros.

  • varchar (n) values are not padded with trailing spaces.

  • varbinary (n) values are not padded with trailing zeros.

When a value is trimmed to fit into a column

  • Trailing spaces are trimmed for char (n) columns.

  • Trailing zeros are trimmed for binary (n) columns.

  • Trailing spaces are not trimmed for varchar (n) columns.

  • Trailing blanks are not trimmed for varbinary (n) columns.

The SQL Server Native Client ODBC driver and SQL Server Native Client OLE DB Provider for SQL Server automatically set ANSI_PADDING to ON when connecting. This can be configured in ODBC data sources, in ODBC connection attributes, or OLE DB connection properties set in the application before connecting.

The SET ANSI_PADDING setting does not affect columns of type nchar or nvarchar. They always display the SET ANSI_PADDING ON behavior. This means trailing spaces and zeros are not trimmed.

The setting of SET ANSI_PADDING is set at execute or run time and not at parse time.

See Also


SET ANSI_WARNINGS (SQL Server PDW)

[image]

SET ANSI_WARNINGS (SQL Server PDW)
See Also Send Feedback

Specifies ISO standard behavior for several error conditions in SQL Server PDW.

In this release, ANSI_WARNINGS is always ON.

For more information, see the SET ANSI_WARNINGS (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET ANSI_WARNINGS ON;

Permissions

Requires membership in the public role.

See Also


SET ARITHABORT (SQL Server PDW)

[image]

SET ARITHABORT (SQL Server PDW)
See Also Send Feedback

Terminates a SQL Server PDW query when an overflow or divide-by-zero error occurs during query execution.

For more information, see the SET ARITHABORT (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET ARITHABORT ON
[ ; ]

Permissions

Requires membership in the public role.

General Remarks

If SET ARITHABORT is ON and SET ANSI WARNINGS is ON, these error conditions cause the query to terminate. Both are always ON in SQL Server PDW, regardless of whether they have been set to ON. As a result, these error conditions will always cause the query to terminate.

Caution note Caution

The default ARITHABORT setting for SQL Server Data Tools for Visual Studio (SSDT) and SQL Server Management Studio is ON. Client applications setting ARITHABORT to OFF can receive different query plans making it difficult to troubleshoot poorly performing queries. That is, the same query can execute fast in management studio but slow in the application. When troubleshooting queries with SSDT or Management Studio always match the client ARITHABORT setting.

Permissions

Requires membership in the public role.

See Also


SET ARITHIGNORE (SQL Server PDW)

[image]

SET ARITHIGNORE (SQL Server PDW)
See Also Send Feedback

Controls whether error messages are returned from overflow or divide-by-zero errors during a SQL Server PDW query. In this release, ARITHIGNORE can only be set to OFF, and therefore SQL Server PDW does not return error messages for these errors.

For more information, see the SET ARITHIGNORE (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET ARITHIGNORE OFF
[ ; ]

Permissions

Requires membership in the public role.

General Remarks

SQL Server PDW returns a NULL in a calculation involving an overflow or divide-by-zero error, regardless of the ARITHIGNORE setting.

This setting does not affect errors occurring during INSERT, UPDATE, and DELETE statements.

The SET ARITHIGNORE setting is set at execute or run time and not at parse time.

Examples

The following example demonstrates the divide by zero and the overflow errors. This example does not return an error message for these errors because ARITHIGNORE is OFF.

Copy image Copy Code
-- SET ARITHIGNORE OFF and testing.
SET ARITHIGNORE OFF;
SELECT 1 / 0 AS DivideByZero;
SELECT CAST(256 AS TINYINT) AS Overflow;

See Also


SET CONCAT_NULL_YIELDS_NULL (SQL Server PDW)

[image]

SET CONCAT_NULL_YIELDS_NULL (SQL Server PDW)
See Also Send Feedback

Specifies that concatenating a string with a NULL value yields a NULL result in SQL Server PDW. CONCAT_NULL_YIELDS_NULL is always ON, and can only be set to ON. For example, SELECT 'abc' + NULL yields NULL. This is useful for backwards compatibility with SQL Server application.

For more information, see SET CONCAT_NULL_YIELDS_NULL (Transact-SQL) on MSDN.

Topic link icon Transact-SQL Syntax Conventions

Syntax

SET CONCAT_NULL_YIELDS_NULL ON

See Also


SET DATEFIRST (SQL Server PDW)

[image]

SET DATEFIRST (SQL Server PDW)
See Also Send Feedback

Sets the first day of the week to a number. In this SQL Server PDW release, 7 is the only valid value, which is Sunday, the U.S. English default.

For more information, see SET DATEFIRST (Transact-SQL)

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET DATEFIRST 7 ;

Permissions

Requires membership in the public role.

General Remarks

To see the current setting of SET DATEFIRST, use the @@DATEFIRST (SQL Server PDW) function.

Example

Copy image Copy Code
-- SET DATEFIRST to U.S. English default value of 7.
SET DATEFIRST 7;

See Also

Reference

@@DATEFIRST (SQL Server PDW)


SET DATEFORMAT (SQL Server PDW)

[image]

SET DATEFORMAT (SQL Server PDW)
See Also Send Feedback

Sets the order of the month, day, and year date parts for interpreting date, smalldatetime, datetime, datetime2 and datetimeoffset character strings. Applies to SQL Server PDW.

In this release, the US default ‘mdy’ is the only valid date format value.

For more information, see the SET DATEFIRST (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET DATEFORMAT { format | @format_var }

Arguments

format | @format_var

Is the order of the date parts. The U.S. English default, mdy, is the only valid parameter.

Permissions

Requires membership in the public role.

Example

Copy image Copy Code
-- Set date format to month/day/year.
SET DATEFORMAT mdy;
DECLARE @datevar datetime2 = '12/31/2012 09:01:01.1234567';
SELECT @datevar;

See Also


SET FMTONLY (SQL Server PDW)

[image]

SET FMTONLY (SQL Server PDW)
See Also Send Feedback

Specifies whether or not a SQL Server PDW query will return only the column header information (e.g., the metadata) without returning any rows of data. Use this setting to view the format of the query results without actually running the query.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET FMTONLY { ON | OFF } [;]

Arguments

ON specifies to return only the column header information. This stays in effect until FMTONLY is set to OFF. The default is OFF, which specifies to return the full results of the query.

Permissions

Requires membership in the public role.

Examples

A. View the column header information for a query without actually running the query.

The following example shows how to return only the column header (metadata) information for a query. The batch begins with FMTONLY set to OFF and changes FMTONLY to ON before the SELECT statement. This causes the SELECT statement to return only the column headers; no rows of data are returned.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
BEGIN
    SET FMTONLY OFF;
    SET DATEFORMAT mdy;
    SET FMTONLY ON;
    SELECT * FROM dbo.DimCustomer;
    SET FMTONLY OFF;
END

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
SET Statements (SQL Server PDW)


SET IMPLICIT_TRANSACTIONS (SQL Server PDW)

[image]

SET IMPLICIT_TRANSACTIONS (SQL Server PDW)
See Also Send Feedback

Sets the implicit transaction mode for the connection to SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET IMPLICIT_TRANSACTIONS { ON | OFF }

General Remarks

When ON, SET IMPLICIT_TRANSACTIONS sets the connection into implicit transaction mode. When OFF, it returns the connection to the AUTOCOMMIT transaction mode.

When a connection is in implicit transaction mode and the connection is not currently in a transaction, executing any of the following statements starts a transaction:

  • ALTER TABLE

  • BEGIN TRANSACTION

  • All CREATE statements

  • DELETE

  • All DROP statements

  • GRANT

  • INSERT

  • REVOKE

  • SELECT

  • UPDATE

If the connection is already in an open transaction, the statements do not start a new transaction.

After a transaction is committed, executing one of the statements above starts a new transaction.

Implicit transaction mode remains in effect until the connection executes a SET IMPLICIT_TRANSACTIONS OFF statement. This returns the connection to AUTOCOMMIT mode, in which all individual statements are committed when they complete successfully.

The SQL Server Native Client OLE DB Provider for SQL Server and the SQL Server Native Client ODBC driver automatically set IMPLICIT_TRANSACTIONS to OFF when connecting.

The IMPLICIT_TRANSACTIONS setting defaults to OFF when connections use the .NET Framework Data Provider for SQL Server (SqlClient) managed provider.

The IMPLICIT_TRANSACTIONS setting is set at run time and not at parse time.

Limitations and Restrictions

Implicit transactions must be explicitly committed or rolled back by the user at the end of the transaction. Otherwise, the transaction and all data changes within the transaction are rolled back when the user disconnects.

See Also

Reference

@@TRANCOUNT (SQL Server PDW)
Transactions (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Connection Strings for JDBC and Microsoft JDBC Driver 4.0 for SQL Server (SQL Server PDW)


SET LOCK_TIMEOUT (SQL Server PDW)

[image]

SET LOCK_TIMEOUT (SQL Server PDW)
See Also Send Feedback

Specifies the number of milliseconds a SQL Server PDW statement waits for a lock to be released. SQL Server PDW always uses the default (-1) behavior for the LOCK_TIMEOUT setting, which is to wait indefinitely for locks to be released.

Note Note

If you set LOCK_TIMEOUT to a value other than -1, SQL Server PDW will ignore the value and continue to wait indefinitely for locks to be released.

For more information, see the SET LOCK_TIMEOUT (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET LOCK_TIMEOUT timeout_period;

Arguments

timeout_period

Is the number of milliseconds that will pass before SQL Server PDW returns a locking error. The default is -1, which indicates to wait forever with no timeout period. All values, other than -1, are ignored.

Permissions

Requires membership in the public role.

SQL Server Differences

SQL Server PDW does not support the READPAST locking hint, which is available in SQL Server as an alternative to using the LOCK_TIMEOUT setting.

Examples

A. Set the lock timeout to wait forever for a lock to be released.

The following example sets the lock timeout to wait forever and never expire. This is the default behavior that is already set at the beginning of each connection.

Copy image Copy Code
SET LOCK_TIMEOUT -1;

The following example sets the lock time-out period to 1800 milliseconds. In this release, SQL Server PDW will parse the statement successfully, but will ignore the value 1800 and continue to use the default behavior.

Copy image Copy Code
SET LOCK_TIMEOUT 1800;

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
SET Statements (SQL Server PDW)


SET NUMERIC_ROUNDABORT (SQL Server PDW)

[image]

SET NUMERIC_ROUNDABORT (SQL Server PDW)
See Also Send Feedback

Specifies the level of error reporting in SQL Server PDW when rounding in an expression causes a loss of precision.

In this release, NUMERIC_ROUNDABORT is always ON.

For more information, see the SET NUMERIC_ROUNDABORT (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET NUMERIC_ROUNDABORT ON;

Permissions

Requires membership in the public role.

General Remarks

When NUMERIC_ROUNDABORT is ON, an error is generated after a loss of precision occurs in an expression.

Loss of precision occurs when an attempt is made to store a value with a fixed precision in a column or variable with less precision.

If NUMERIC_ROUNDABORT is ON, the ARITHABORT setting determines the severity of the generated error. Since NUMERIC_ROUNDABORT and ARITHABORT are always ON, SQL Server PDW will generate an error for precision loss and will not return a result set.

The setting of SET NUMERIC_ROUNDABORT is set at execute or run time and not at parse time.

See Also


SET QUOTED_IDENTIFIER (SQL Server PDW)

[image]

SET QUOTED_IDENTIFIER (SQL Server PDW)
See Also Send Feedback

Causes SQL Server PDW to follow the ISO rules regarding quotation mark delimiting identifiers and literal strings. Use this setting to allow identifiers to be reserved keywords or to contain characters that are not allowed by the syntax rules for identifiers.

In this release, the QUOTED_IDENTIFIER setting is always ON.

For more information, see the SET QUOTED_IDENTIFIER (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET QUOTED_IDENTIFIER ON

Permissions

Requires membership in the public role.

General Remarks

When SET QUOTED_IDENTIFIER is ON:

  1. Identifiers can be delimited by double quotation marks.

  2. Literal strings must be delimited by single quotation marks. Literal strings cannot be delimited by double quotation marks.

  3. If a single quotation mark (') is part of the literal string, it can be represented by two single quotation marks (").

  4. All strings delimited by double quotation marks are interpreted as object identifiers

  5. Quoted identifiers do not have to follow the SQL rules for identifiers. They can be reserved keywords and can include characters not generally allowed in Transact-SQL identifiers.

SET QUOTED_IDENTIFIER is set at parse time. Setting at parse time means that if the SET statement is present in the batch or stored procedure, it takes effect, regardless of whether code execution actually reaches that point; and the SET statement takes effect before any statements are executed.

Using brackets, [ and ], to delimit identifiers is not affected by the QUOTED_IDENTIFIER setting.

See Also


SET ROWCOUNT (SQL Server PDW)

[image]

SET ROWCOUNT (SQL Server PDW)
See Also Send Feedback

Causes SQL Server PDW to stop processing the query after the specified number of rows are returned.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET ROWCOUNT { number | @number_var }
[;]

Arguments

number | @number_var

An integer number of rows to be returned by each query in the session.

General Remarks

SET ROWCOUNT 0 turns the option off so that the query will return all rows.

When ROWCOUNT is set and TOP is in the SELECT statement, the query returns the smaller number of rows. For example, the following query will return at most 3 rows.

Copy image Copy Code
SET ROWCOUNT 10;
SELECT TOP 3 * FROM sys.databases;

SET ROWCOUNT is a run time setting, and stays in effect for the session or until it is changed.

Limitations and Restrictions

SET ROWCOUNT does not affect these statements:

  • DELETE (SQL Server PDW)

  • INSERT (SQL Server PDW)

  • UPDATE (SQL Server PDW)

  • CREATE TABLE AS SELECT (SQL Server PDW)

  • CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW)

  • CREATE REMOTE TABLE AS SELECT (SQL Server PDW)

For a similar behavior, use the TOP (SQL Server PDW) statement.

Permissions

Requires membership in the public role.

Examples

SET ROWCOUNT stops processing after the specified number of rows. In the following example, note that more than 20 rows meet the criteria of AccountType = 'Assets'. However, after applying SET ROWCOUNT, you can see that not all rows were returned.

Copy image Copy Code
USE AdventureWorksPDW2012;

SET ROWCOUNT 5;
SELECT * FROM [dbo].[DimAccount]
WHERE AccountType = 'Assets';

To return all rows, set ROWCOUNT to 0.

Copy image Copy Code
USE AdventureWorksPDW2012;

SET ROWCOUNT 0;
SELECT * FROM [dbo].[DimAccount]
WHERE AccountType = 'Assets';

See Also


SET TEXTSIZE (SQL Server PDW)

[image]

SET TEXTSIZE (SQL Server PDW)
See Also Send Feedback

Specifies the size of varchar(max), nvarchar(max), and varbinary(max) data returned by a SELECT statement in SQL Server PDW.

These types are not directly supported by the SQL Server PDW SQL language, but could be displayed as the result of querying the SQL Server system views.

For more information, see the SET TEXTSIZE (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET TEXTSIZE number ;

Arguments

number

Is the length of varchar(max), nvarchar(max), or varbinary(max) data, in bytes. number is an integer. The maximum setting for SET TEXTSIZE is 2 gigabytes (GB), specified in bytes. A setting of 0 resets the size to the default (4 KB).

Permissions

Requires membership in the public role.

General Remarks

The SQL Server Native Client ODBC driver and SQL Server Native Client OLE DB Provider for SQL Server automatically set TEXTSIZE to 2147483647 when connecting.

TEXTSIZE is set at run time and not at parse time.

See Also


SET TRANSACTION ISOLATION LEVEL (SQL Server PDW)

[image]

SET TRANSACTION ISOLATION LEVEL (SQL Server PDW)
See Also Send Feedback

Controls the locking and row versioning behavior of SQL statements issued by a connection to SQL Server PDW.

In this release, the isolation level is always READ UNCOMMITTED.

For more information, see the SET TRANSACTION ISOLATION LEVEL (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
[ ; ]

Arguments

READ UNCOMMITTED

Specifies that statements can read rows that have been modified by other transactions but not yet committed.

See Also


SET XACT_ABORT (SQL Server PDW)

[image]

SET XACT_ABORT (SQL Server PDW)
See Also Send Feedback

Specifies whether SQL Server PDW automatically rolls back the current transaction when a SQL statement raises a run-time error.

In this release, SET XACT_ABORT is always ON.

For more information, see the SET XACT_ABORT (Transact-SQL) documentation on MSDN.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

SET XACT_ABORT ON

General Remarks

When SET XACT_ABORT is ON, if a SQL statement raises a run-time error, the entire transaction is terminated and rolled back.

SET XACT_ABORT is set at execute or run time and not at parse time.

See Also


Data Definition Statements (SQL Server PDW)

[image]

Data Definition Statements (SQL Server PDW)
See Also Send Feedback

SQL Server PDW includes data definition statements for manipulation of tables, indexes, and other database object

This topic is organized by object type, as shown:

Database Management

Table, Index, and Statistics Management

View Management

Database Management

Command | Description
ALTER AUTHORIZATION | Changes ownership of the database.
ALTER DATABASE | Modifies the maximum size options for replicated tables, distributed tables, and the transaction log in the database.
BACKUP DATABASE | Backs up a database to the Backup server.
CREATE DATABASE | Creates a new database.
DROP DATABASE | Removes a database.
RENAME | Renames a database.
RESTORE DATABASE | Restores a database created with BACKUP DATABASE.
USE | Specifies the database context.

Table, Index, and Statistics Management

Statement | Description
ALTER AUTHORIZATION | Changes the ownership of a table.
ALTER INDEX | Rebuilds indexes on a table.
ALTER TABLE | Splits, merges, or switches table partitions.
CREATEXTERNAL DATA SOURCE | Creates a data source for an external Hadoop table.
CREATEXTERNAL FILE FORMAT | Creates a defined file format for an external Hadoop table.
CREATE INDEX | Creates a non-clustered index on a table.
CREATE STATISTICS | Creates query optimization statistics on a table.
CREATE EXTERNAL TABLE | Creates an external table that points to a Hadoop cluster for the table data.
CREATE EXTERNAL TABLE AS SELECT | Creates an external table on a Hadoop cluster and populates it with the results of a select statement.
CREATE REMOTE TABLE | Copies data to a new table in a non-appliance SQL Server database.
CREATE TABLE | Creates a new table.
CREATE TABLE AS SELECT | Creates a table that contains the results of a select statement.
DROP INDEX | Removes and deletes a non-clustered index.
DROP STATISTICS | Removes and deletes statistics from a table.
DROP TABLE | Removes and deletes a table.
RENAME | Renames a table.
UPDATE STATISTICS | Updates statistics.

View Management

Statement | Description
CREATE VIEW | Creates a view.
DROP VIEW | Removes and deletes a view.

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Security Statements (SQL Server PDW)
Data Manipulation Statements (SQL Server PDW)
Queries (SQL Server PDW)


ALTER DATABASE (SQL Server PDW)

[image]

ALTER DATABASE (SQL Server PDW)
See Also Send Feedback

Modifies the maximum database size options for replicated tables, distributed tables, and the transaction log in SQL Server PDW. Use this statement to manage disk space allocations for a database as it grows or shrinks in size.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER DATABASE database_name
    SET ( <set_database_options>   | <db_encryption_option> )
[;]

<set_database_options> ::=
{
    AUTOGROW = { ON | OFF }
    | REPLICATED_SIZE = size [GB]
    | DISTRIBUTED_SIZE = size [GB]
    | LOG_SIZE = size [GB]
}

<db_encryption_option> ::=
    ENCRYPTION { ON | OFF }

Arguments

database_name

The name of the database to be modified. To display a list of databases on the appliance, use sys.databases (SQL Server PDW).

AUTOGROW = { ON | OFF }

Updates the AUTOGROW option. When AUTOGROW is ON, SQL Server PDW automatically increases the allocated space for replicated tables, distributed tables, and the transaction log as necessary to accommodate growth in storage requirements. When AUTOGROW is OFF, SQL Server PDW returns an error if replicated tables, distributed tables, or the transaction log exceeds the maximum size setting.

REPLICATED_SIZE = size [GB]

Specifies the new maximum gigabytes per Compute node for storing all of the replicated tables in the database being altered. If you are planning for appliance storage space, you will need to multiply REPLICATED_SIZE by the number of Compute nodes in the appliance.

DISTRIBUTED_SIZE = size [GB]

Specifies the new maximum gigabytes per database for storing all of the distributed tables in the database being altered. The size is distributed across all of the Compute nodes in the appliance.

LOG_SIZE = size [GB]

Specifies the new maximum gigabytes per database for storing all of the transaction logs in the database being altered. The size is distributed across all of the Compute nodes in the appliance.

ENCRYPTION { ON | OFF }

Sets the database to be encrypted (ON) or not encrypted (OFF). Encryption can only be configured for SQL Server PDW when sp_pdw_database_encryption has been set to 1. A database encryption key must be created before transparent data encryption can be configured. For more information about database encryption, see Transparent Data Encryption (SQL Server PDW).

Permissions

Requires the ALTER permission on the database.

General Remarks

The values for REPLICATED_SIZE, DISTRIBUTED_SIZE, and LOG_SIZE can be greater than, equal to, or less than the current values for the database.

Limitations and Restrictions

Grow and shrink operations are approximate. The resulting actual sizes can vary from the size parameters.

SQL Server PDW does not perform the ALTER DATABASE statement as an atomic operation. If the statement is aborted during execution, changes that have already occurred will remain.

Locking Behavior

Takes a shared lock on the DATABASE object. You cannot alter a database that is in use by another user for reading or writing. This includes sessions that have issued a USE statement on the database.

Performance

Shrinking a database can take a large amount of time and system resources, depending on the size of the actual data within the database, and the amount of fragmentation on disk. For example, shrinking a database could take serveral hours or more.

Determining Encryption Progress

Use the following query to determine progress of database transparent data encryption as a percent:

Copy image Copy Code
WITH
database_dek AS (
    SELECT ISNULL(db_map.database_id, dek.database_id) AS database_id,
        dek.encryption_state, dek.percent_complete,
        dek.key_algorithm, dek.key_length, dek.encryptor_thumbprint,
        type
    FROM sys.dm_pdw_nodes_database_encryption_keys AS dek
    INNER JOIN sys.pdw_nodes_pdw_physical_databases AS node_db_map
        ON dek.database_id = node_db_map.database_id
        AND dek.pdw_node_id = node_db_map.pdw_node_id
    LEFT JOIN sys.pdw_database_mappings AS db_map
        ON node_db_map .physical_name = db_map.physical_name
    INNER JOIN sys.dm_pdw_nodes nodes
        ON nodes.pdw_node_id = dek.pdw_node_id
    WHERE dek.encryptor_thumbprint <> 0x
),
dek_percent_complete AS (
    SELECT database_dek.database_id, AVG(database_dek.percent_complete) AS percent_complete
    FROM database_dek
    WHERE type = 'COMPUTE'
    GROUP BY database_dek.database_id
)
SELECT DB_NAME( database_dek.database_id ) AS name,
    database_dek.database_id,
    ISNULL(
       (SELECT TOP 1 dek_encryption_state.encryption_state
        FROM database_dek AS dek_encryption_state
        WHERE dek_encryption_state.database_id = database_dek.database_id
        ORDER BY (CASE encryption_state
            WHEN 3 THEN -1
            ELSE encryption_state
            END) DESC), 0)
        AS encryption_state,
dek_percent_complete.percent_complete,
database_dek.key_algorithm, database_dek.key_length, database_dek.encryptor_thumbprint
FROM database_dek
INNER JOIN dek_percent_complete
    ON dek_percent_complete.database_id = database_dek.database_id
WHERE type = 'CONTROL';

For a comprehensive example demonstrating all the steps in implementing TDE, see Transparent Data Encryption (SQL Server PDW).

Examples

A. Altering the AUTOGROW setting

Set AUTOGROW to ON for database CustomerSales.

Copy image Copy Code
ALTER DATABASE CustomerSales
    SET ( AUTOGROW = ON );

B. Altering the maximum storage for replicated tables

The following example sets the replicated table storage limit to 1 GB for the database CustomerSales. This is the storage limit per Compute node.

Copy image Copy Code
ALTER DATABASE CustomerSales
    SET ( REPLICATED_SIZE = 1 GB );

C. Altering the maximum storage for distributed tables

The following example sets the distributed table storage limit to 1000 GB (one terabyte) for the database CustomerSales. This is the combined storage limit across the appliance for all of the Compute nodes, not the storage limit per Compute node.

Copy image Copy Code
ALTER DATABASE CustomerSales
    SET ( DISTRIBUTED_SIZE = 1000 GB );

D. Altering the maximum storage for the transaction log

The following example updates the database CustomerSales to have a maximum SQL Server transaction log size of 10 GB for the appliance.

Copy image Copy Code
ALTER DATABASE CustomerSales
    SET ( LOG_SIZE = 10 GB );

See Also

Reference

CREATE DATABASE (SQL Server PDW)
DROP DATABASE (SQL Server PDW)


ALTER INDEX (SQL Server PDW)

[image]

ALTER INDEX (SQL Server PDW)
See Also Send Feedback

Rebuilds or reorganizes an index or index partition on a SQL Server PDW table. This can also disable an index.

Use ALTER INDEX REBUILD to:

  • Defragment a rowstore or columnstore index or an index partition.

  • Merge rowstore and columnstore data into the columnstore for a clustered columnstore index.

  • Reduce the storage size of a fragmented clustered columnstore index.

Use ALTER INDEX REORGANIZE to:

  • Improve query performance for a clustered columnstore index by moving CLOSED rowgroups into the columnstore.

  • Reorganize the data in the leaf level of a rowstore index while keeping the index online.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER INDEX {index_name | ALL }
    ON [ database_name . [ schema_name ] . | schema_name. ] table_name
{
    REBUILD [ PARTITION = { ALL | partition_number } ]
    | DISABLE
    | REORGANIZE [ PARTITION = partition_number ]
}
[;]

Arguments

index_name | ALL

Specifies the name of the index. ALL rebuilds all indexes on the table.

[ database_name . [ schema_name ] . | schema_name . ] table_name

Specifies the one-, two-, or three-part name of the table that contains the index to alter. table_name can also be a temporary table.

REBUILD [ PARTITION = { ALL | partition_number } ]

Specifies to rebuild the index without changing the index definition.

REBUILD

Specifies to rebuild the entire index.

To rebuild a clustered columnstore index, SQL Server PDW:

  • Acquires an exclusive lock on the table or partition while the rebuild occurs. The data is “offline” and unavailable during the rebuild.

  • Defragments the columnstore by physically deleting rows that have been logically deleted from the table; the deleted bytes are reclaimed on the physical media.

  • Merges the rowstore data, stored in the deltastore, with the data in the columnstore before it rebuilds the index. When the rebuild is finished, all data is stored in columnstore format, and the deltastore, which is a rowstore, contains 0 rows.

  • Re-compresses all data into the columnstore. Two copies of the columnstore index exist while the rebuild is taking place. When the rebuild is finished, SQL Server deletes the original columnstore index.

REBUILD PARTITION = ALL

Rebuilds all partitions of the index.

REBUILD PARTITION = partition_number

Rebuilds only a specific partition of the index. partition_number identifies the number of the partition to rebuild.

For a clustered columnstore index, REBUILD and REBUILD PARTITION = ALL accomplish the same results.

DISABLE

Marks the index as disabled and unavailable for use. Any index can be disabled. When an index is disabled, the index definition remains in the system catalog and is visible in the catalog views, however, all index data gets deleted. The table data is not deleted, only the index data is deleted.

After disabling a clustered index, you cannot access the underlying table data until the index is enabled. Disabling a clustered columnstore index causes all table data to be unavailable until the index is enabled.

To enable an index use ALTER INDEX REBUILD or CREATE INDEX (SQL Server PDW) with the DROP_EXISTING clause.

REORGANIZE [ PARTITION = partition_number ]

For clustered columnstore indexes, this specifies to move all CLOSED rowgroups into the columnstore.

For rowstore indexes, this specifies to reorganize the data in the leaf level of the index while keeping the index online. Operations on the table, such as updates and queries, can continue while an index is being reorganized. Reorganizing an index is useful for indexes that have a small amount of defragmentation.

The REORGANIZE operation cannot be specified for a disabled index.

ALLOW_PAGE_LOCKS is always ON in SQL Server PDW and will not block you from disabling an index.

PARTITION = partition_number

Reorganizes only the partition numbered as partition_number.

Permissions

Requires ALTER permission on the table, or membership in one of the following roles:

  • sysadmin fixed server role

  • db_ddladmin fixed database role

  • db_owner fixed database role.

General Remarks

When to reorganize a clustered columnstore index:

  • Reorganize a clustered columnstore index after one or more data loads to achieve query performance benefits as quickly as possible. Reorganizing will initially require additional CPU resources to compress the data, which could slow overall system performance. However, as soon as the data is compressed, query performance can improve.

Reorganizing is not required for moving CLOSED rowgroups into the columnstore. The tuple-mover process will eventually find all CLOSED rowgroups and move them. However, the tuple-mover is single-threaded and might not move rowgroups fast enough for your workload.

When to rebuild a clustered columnstore index:

  • Use caution before rebuilding an entire clustered columnstore index because this takes a long time if the index is large, and it requires enough disk space to store an additional copy of the index while it is rebuilding. Usually it is only necessary to rebuild the most recently used partition

  • Rebuild a partition after heavy DML operations to defragment the index and reduce disk storage. For example, deleting rows marks them for deletion but does not actually delete the row from the columnstore until the index is rebuilt. Delting rows also occurs during an update operation; to update a row, SQL Server PDW inserts a new row, and then marks the old row for deletion.

  • Rebuild a partition after loading data to ensure all data is stored in the columnstore. This is useful, especially if the loaded rows land in one or more deltastores of the individual distributions. Note that if multiple loads occur at the same time, each distribution could end up having multiple deltastores.

ALTER INDEX runs in parallel across the Compute nodes. For distributed tables, ALTER INDEX also runs in parallel across the distributions on each Compute node.

Locking Behavior

Takes an exclusive lock on the TABLE object. Takes a shared lock on the DATABASE object.

You cannot rebuild an index when the index or the table that contains the index is in use by another user.

REBUILD runs as an offline operation.

REORGANIZE runs as an online operation. While the index is reorganizing, reads are allowed on the index, and writes are not allowed.

Examples

A. Basic Syntax

ALTER INDEX index1 ON table1 REBUILD;

ALTER INDEX ALL ON table1 REBUILD;

ALTER INDEX ALL ON dbo.table1 REBUILD;

See Also

Reference

CREATE INDEX (SQL Server PDW)
DROP INDEX (SQL Server PDW)


ALTER PROCEDURE (SQL Server PDW)

[image]

ALTER PROCEDURE (SQL Server PDW)
See Also Send Feedback

Changes the definition of a stored procedure on a SQL Server PDW database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

Transact-SQL
--Transact-SQL Stored Procedure Syntax
ALTER { PROC | PROCEDURE } [schema_name.] procedure_name
    [ { @parameter data_type } [= ] ] [ ,...n ]
AS { [ BEGIN ] sql_statement [ ; ] [ ,...n ] [ END ] }
[;]

Arguments

schema_name

The name of the schema to which the procedure belongs.

procedure_name

The name of the procedure to change. Procedure names must comply with the rules for identifiers.

@parameter

A parameter in the procedure. Up to 2,100 parameters can be specified.

{ [ BEGIN ] sql_statement [;] [ ...n ] [ END ] }

One or more Transact-SQL statements comprising the body of the procedure. You can use the optional BEGIN and END keywords to enclose the statements. For more information, see the Remarks section in CREATE PROCEDURE (SQL Server PDW).

General Remarks

For a general description of stored procedures, see the SQL Server documentation at CREATE PROCEDURE (Transact-SQL).

Permissions

Requires ALTER permission on the stored procedure, or membership in the db_ddladmin fixed database role.

The sql_statement section of the stored procedure may require additional permissions.

Locking

Takes an exclusive lock on the procedure. Takes a shared lock on the DATABASE, SCHEMA, and SCHEMARESOLUTION object.

See Also

Reference

CREATE PROCEDURE (SQL Server PDW)
DROP PROCEDURE (SQL Server PDW)


ALTER SCHEMA (SQL Server PDW)

[image]

ALTER SCHEMA (SQL Server PDW)
See Also Send Feedback

Transfers a securable between schemas.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER SCHEMA schema_name
   TRANSFER [ OBJECT :: ] securable_name
[;]

Arguments

schema_name

Is the name of a schema in the current database, into which the securable will be moved. Cannot be SYS or INFORMATION_SCHEMA.

OBJECT

Is the class of the entity for which the owner is being changed. Object is the only class supported in PDW. Tables, views, and procedures are all in the object class.

securable_name

Is the one-part or two-part name of a schema-contained securable to be moved into the schema.

Remarks

Users and schemas are completely separate.

ALTER SCHEMA can only be used to move securables between schemas in the same database. To change or drop a securable within a schema, use the ALTER or DROP statement specific to that securable.

If a one-part name is used for securable_name, the name-resolution rules currently in effect will be used to locate the securable.

All permissions associated with the securable will be dropped when the securable is moved to the new schema. If the owner of the securable has been explicitly set, the owner will remain unchanged. If the owner of the securable has been set to SCHEMA OWNER, the owner will remain SCHEMA OWNER; however, after the move SCHEMA OWNER will resolve to the owner of the new schema. The principal_id of the new owner will be NULL.

Permissions

To transfer a securable from another schema, the current user must have CONTROL permission on the securable (not schema) and ALTER permission on the target schema.

If the securable has an EXECUTE AS OWNER specification on it and the owner is set to SCHEMA OWNER, the user must also have IMPERSONATION permission on the owner of the target schema.

All permissions associated with the securable that is being transferred are dropped when it is moved.

Locking

Takes an exclusive lock on the SCHEMA. Takes a shared lock on the DATABASE object.

Examples

A. Transferring ownership of a table

The following example creates a table Region in the dbo schema, creates a Sales schema, and then moves the Region table from the dbo schema to the Sales schema.

Copy image Copy Code
CREATE TABLE dbo.Region
    (Region_id int NOT NULL,
    Region_Name char(5) NOT NULL)
WITH (DISTRIBUTION = REPLICATE);
GO

CREATE SCHEMA Sales;
GO

ALTER SCHEMA Sales TRANSFER OBJECT::dbo.Region;
GO

See Also

Reference

CREATE SCHEMA (SQL Server PDW)
DROP SCHEMA (SQL Server PDW)
sys.schemas (SQL Server PDW)
Functions (SQL Server PDW)


ALTER TABLE (SQL Server PDW)

[image]

ALTER TABLE (SQL Server PDW)
See Also Send Feedback

Modifies the definition for a table in SQL Server PDW. Use ALTER TABLE to:

  • Modify a column definition.

  • Add or drop a column or column constraint.

  • Split, merge, or switch table partitions in a SQL Server PDW database. Table partitions are often used to manage data storage and archival.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER TABLE [ database_name . [schema_name ] . | schema_name. ] source_table_name
{
    ALTER COLUMN column_name
        {
            type_name [ ( precision [ , scale ] ) ]
            [ COLLATE Windows_collation_name ]
            [ NULL | NOT NULL ]
        }
    | ADD { <column_definition> | <column_constraint> FOR column_name} [ ,...n ]
    | DROP { COLUMN column_name | [CONSTRAINT] constraint_name } [ ,...n ]
    | REBUILD [ PARTITION = { ALL | partition_number } ]
    | { SPLIT | MERGE } RANGE (boundary_value)
    | SWITCH [ PARTITION source_partition_number
        TO target_table_name [ PARTITION target_partition_number ]
}
[;]

<column_definition>::=
{
    column_name
    type_name [ ( precision [ , scale ] ) ]
    [ <column_constraint> ]
    [ COLLATE Windows_collation_name ]
    [ NULL | NOT NULL ]
}

<column_constraint>::=
    [ CONSTRAINT constraint_name ] DEFAULT constant_expression

Arguments

database_name

Specifies the name of the database that contains the table to update.

schema_name

Specifies the schema of the table to update. If omitted, the default is the default schema of the user, or dbo.

source_table_name

Specifies the name of the table to modify.

When using:

  • ALTER COLUMN, the source table is the table to modify. The source table cannot be a temporary table.

  • ADD, the source table will receive the added column.

  • DROP, the source table will relinquish the dropped column.

  • SPLIT or MERGE, the source table will receive the partition updates.

  • SWITCH, the source table contains the data that will be moved to another table.

ALTER COLUMN

Modifies the column definition for an existing column.

You cannot use ALTER COLUMN to modify a distribution column; if you need to do this, you can copy the data into a new table by using CREATE TABLE AS SELECT (SQL Server PDW).

You cannot change the data type, length, precision, or scale of a column that has any of the following characteristics:

  • Column is included in an index.

  • Column is in a partitioned table.

  • Column is included in a user-defined statistics object.

For a column that has a DEFAULT constraint:

  • You cannot change the data type.

  • You can change the length, precision, or scale.

Some data type changes can cause the data to change. For example:

  • Changing the data type from Unicode (nchar, nvarchar) to ASCII (char, varchar) can cause characters in the extended character set to be converted to ASCII.

  • Reducing the length, precision, or scale of a column can cause data truncation.

  • Converting a column of type FLOAT or REAL to a different data type can cause inconsistencies in data across replicated tables. Likewise, replicated tables can have inconsistencies in data when a column of a different type is converted to FLOAT or REAL.

SQL Server PDW will drop all auto-generated statistics on the modified column.

column_name

For ALTER COLUMN, column_name specifies to modify the definition for an existing column in the source table. The column to be modified cannot be a distribution column.

For ADD, column_name specifies the name of the column to add.

type_name [ ( precision [ , scale ] ) ]

Specifies the new data type for the column.

  • The data type must be one of the supported SQL Server PDW data types. For a complete list of data types, see the data type options and descriptions in CREATE TABLE (SQL Server PDW).

  • SQL Server PDW must support implicit conversions from the current data type to the new data type. For implicit type conversion rules, use the SQL Server rules. See the Implicit Conversions section in CAST and CONVERT (Transact-SQL). When reading the SQL Server topic, ignore data types that are not supported by SQL Server PDW.

  • For more information about valid precision and scale values, see Precision, Scale, and Length (Transact-SQL).

COLLATE Windows_collation_name

Specifies the new collation for the modified column. If not specified, the column is assigned the collation of the database.

The collation name must be a Windows collation name. For a list of Windows collations, see Windows Collation Name (Transact-SQL).

To use the collation clause, the source column:

  • Must be of type char, varchar, nchar, or nvarchar.

  • Cannot be part of an index.

  • Cannot be included in any user-defined statistics. The column can have auto-generated statistics. All auto-generated statistics will be dropped on the column receiving the collation change, which is the source column.

For more information about collations in SQL Server PDW, see Collations (SQL Server PDW).

For more information about column definitions in SQL Server PDW, see CREATE TABLE (SQL Server PDW).

NULL | NOT NULL

Defines whether or not column values can be set to NULL.

[ CONSTRAINT constraint_name ] DEFAULT constant_expression

Specifies the default column value.

constraint_name

The optional name for the constraint. The constraint name is unique within the database. The name can be re-used in other databases.

constant_expression

The default value for the column. The expression must be a literal value or a constant.

Constant expression examples - allowed:

  • ‘CA’

  • 4

Constant expression examples – not allowed:

  • ‘CA’ + ‘1’

  • 2+3

  • CURRENT_TIMESTAMP

Restrictions:

  • You cannot define a DEFAULT constraint on a distribution column.

  • You cannot modify a DEFAULT constraint. If you need to change the value for a default constraint, you can drop and recreate the constraint. This update is quick because the change does not modify existing data. This change only affects new rows to be added. Recreating the constraint does not change old rows.

ADD { <column_definition> | <column_constraint> FOR COLUMN column_name} [ ,...n ]

Specifies to add a column or a column constraint. Multiple columns and constraints are allowed.

For more information about defining a column or a column constraint, see CREATE TABLE (SQL Server PDW)

DROP { [CONSTRAINT] constraint_name | COLUMN column_name } [ ,...n ]

Specifies to remove a column constraint or a column from the table. One or more constraints and columns can be listed.

You cannot drop a column that:

  • Is a distribution column.

  • Is included in an index.

  • Has a DEFAULT constraint.

  • Is the partition column in a partitioned table. .

REBUILD [ PARTITION = { ALL | partition_number } ]

Specifies to rebuild the table and to rebuild all clustered indexes on the table. This is useful for defragmenting the table data.

Rebuilding a columnstore table can reduce the storage required for the table. Some columnstore data is temporarily stored in rowstore format. Rebuilding the table will bring all of the data into the columnstore.

Rebuild is an online operation for rowstore tables, and a partially offline operation for columnstore tables. During the rebuild of a columnstore table, the table data can be read, but cannot be changed.

REBUILD

Rebuild the entire table, including all partitions.

REBUILD PARTITION = ALL

Rebuilds all partitions of the columns in the index .

REBUILD PARTITION = partition_number

Rebuilds the table partition numbered as partition_number.

{ SPLIT | MERGE } RANGE (boundary_value)
SPLIT

Add a new partition by splitting one partition into two partitions. When a columnstore index exists on the table only empty partitions can be split.

MERGE

Remove a partition by combining two partitions into one partition.

RANGE (boundary_value)

When splitting a partition, boundary_value specifies the new partition boundary to be added. When merging two partitions, boundary_value specifies the partition boundary to be removed.

SWITCH [ PARTITION source_partition_number ] TO target_table_name [ PARTITION target_partition_number ]

Move a partition from one table to another table.

PARTITION source_partition_number

Specifies a partition to switch from the source table. When the PARTITION clause is specified, the source table must be a partitioned table. When the PARTITION clause is omitted, the source table must be a non-partitioned table and all table rows will be switched.

source_partition_number is the number of the partition to switch. This is a number from 1 to the number of partitions in source_table_name. Specify source_partition_number as a constant.

target_table_name

The table that logically receives the moved data. The source and target table must be different tables within the same database.

PARTITION target_partition_number

PARTITION specifies to logically move the source table data to a partition in the target table. When the PARTITION clause is specified, the target table must be a partitioned table and the target partition must be empty.

target_partition_number is the number of the partition that will receive the data. This is a number from 1 to the number of partitions in the target table. Specify target_partition_number as a constant.

The SWITCH statement allows or disallows the following source and destination variations:

  • A table partition can be switched to a table partition in another table.

  • A table partition can be switched to a non-partitioned table.

  • A non-partitioned table can be switched to a non-partitioned table.

  • A non-partitioned table cannot be switched to a partitioned table.

Requirements for a partition switch operation:

  • The source and target tables must have identical distribution key columns, column definitions, column order, clustered indexes, and non-clustered indexes.

  • If the source and target tables are both partitioned, they must be partitioned on the same column.

  • The source table and the target table must both be rowstore tables or both be columnstore tables. You cannot switch a partition from a rowstore table to a columnstore table, nor from a columnstore table to a rowstore table.

  • The source table and the target table must both be replicated or both be distributed. You cannot switch a partition from a distributed table to a replicated table, nor from a replicated table to a distributed table.

  • When switching from a source partition to a target partition, the range of boundary values for the source partition must be within the range of boundary values for the target partition.

  • Before performing the SWITCH operation, the target partition or target non-partitioned table must be empty.

Permissions

Requires ALTER permission on the table or membership in the db_ddladmin fixed database role.

SWITCH operations require ALTER permission on both the source and target tables.

SPLIT and MERGE operations require the ALTER ANY DATASPACE permission.

Adding a column that updates the rows of the table requires UPDATE permission on the table. For example, adding a NOT NULL column with a default value.

Error Handling

The ALTER TABLE statement cannot be used within a transaction, but is transactionally safe. If you cancel an ALTER TABLE statement or have a run-time failure, the ALTER TABLE statement will fail and all changes will be rolled back. All data will remain intact with no changes.

General Remarks

Indexes are always partitioned in the same manner as the table. Therefore, when altering table partitions, SQL Server PDW also alters the indexes to align with the altered partitions.

Limitations and Restrictions

The ALTER TABLE statement cannot be used within a transaction.

ALTER TABLE cannot be used with the EXPLAIN command.

Locking Behavior

Takes an exclusive lock on the table. Takes a shared lock on the DATABASE, and SCHEMA objects. ADD, DROP, and ALTER are offline operations. All read and write requests will be blocked until the operation finished. This includes DDL and DML operations on the table and on the database that contains the table.

REBUILD is an online operation for rowstore tables, and a partially offline operation for columnstore tables. During the rebuild of a columnstore table, the table data is readable, but not writeable.

ALTER TABLE PARTITION takes a shared lock on the SCHEMARESOLUTION object.

Metadata

To view the system-supplied name of constraints, use the sys.default_constraints (SQL Server PDW) catalog view.

To view partition metadata, you can use the sys.partition_range_values (SQL Server PDW) catalog view.

Performance Considerations

Use caution when running ALTER TABLE to add, modify, or drop columns. ALTER TABLE can take a long time to run, especially when a large number of rows need to be updated.

Performance tips:

  • Modifying an existing column is much slower than adding a new column.

  • Adding a NOT NULL column with a DEFAULT constraint is IO intensive because the system generates many log records.

  • Conversely, dropping a column is a quick operation because the operation is performed in the metadata.

  • Using CREATE TABLE AS SELECT (SQL Server PDW) to modify a table in lieu of ALTER TABLE ALTER could improve performance in some situations.

The SPLIT and MERGE operations are usually IO intensive, unless they are applied to empty partitions, because they move the physical data among partitions. It is a best practice to perform SPLIT and MERGE operations on empty partitions.

The SWITCH operation is not IO intensive because it moves only metadata and does not move the physical data.

For replicated tables, adding a column executes in parallel across the Compute nodes. For distributed tables, adding a column executes in parallel across the Compute nodes and serially across the distributions within each Compute node. All other alter column or drop column operations are executed in parallel across the Compute nodes and serialized across the distributions within a Compute node.

All read and write operations on the table, and all DDL or DML operations on the database, will block when ALTER COLUMN operations are executing.

Examples

The following examples A through C use the FactResellerSales table in the AdventureWorksPDW2012 database.

A. Determining if a table is partitioned

The following query returns one or more rows if the table FactResellerSales is partitioned. If the table is not partitioned, no rows are returned.

Copy image Copy Code
SELECT * FROM sys.partitions AS p
JOIN sys.tables AS t
    ON  p.object_id = t.object_id
WHERE p.partition_id IS NOT NULL
    AND t.name = 'FactResellerSales';

B. Determining boundary values for a partitioned table

The following query returns the boundary values for each partition in the FactResellerSales table.

Copy image Copy Code
SELECT t.name AS TableName, i.name AS IndexName, p.partition_number, p.partition_id, i.data_space_id, f.function_id, f.type_desc, r.boundary_id, r.value AS BoundaryValue
FROM sys.tables AS t
JOIN sys.indexes AS i
    ON t.object_id = i.object_id
JOIN sys.partitions AS p
    ON i.object_id = p.object_id AND i.index_id = p.index_id
JOIN  sys.partition_schemes AS s
    ON i.data_space_id = s.data_space_id
JOIN sys.partition_functions AS f
    ON s.function_id = f.function_id
LEFT JOIN sys.partition_range_values AS r
    ON f.function_id = r.function_id and r.boundary_id = p.partition_number
WHERE t.name = 'FactResellerSales' AND i.type <= 1
ORDER BY p.partition_number;

C. Determining the partition column for a partitioned table

The following query returns the name of the partitioning column for table. FactResellerSales.

Copy image Copy Code
SELECT t.object_id AS Object_ID, t.name AS TableName, ic.column_id as PartitioningColumnID, c.name AS PartitioningColumnName
FROM sys.tables AS t
JOIN sys.indexes AS i
    ON t.object_id = i.object_id
JOIN sys.columns AS c
    ON t.object_id = c.object_id
JOIN sys.partition_schemes AS ps
    ON ps.data_space_id = i.data_space_id
JOIN sys.index_columns AS ic
    ON ic.object_id = i.object_id AND ic.index_id = i.index_id AND ic.partition_ordinal > 0
WHERE t.name = 'FactResellerSales'
AND i.type <= 1
AND c.column_id = ic.column_id;

D. Merging two partitions

The following example merges two partitions on a table.

The Customer table has the following definition:

Copy image Copy Code
CREATE TABLE Customer (
    id int NOT NULL,
    lastName varchar(20),
    orderCount int,
    orderDate date)
WITH
    ( DISTRIBUTION = HASH(id),
    PARTITION ( orderCount RANGE LEFT
    FOR VALUES (1, 5, 10, 25, 50, 100)));

The following command combines the 10 and 25 partition boundaries.

Copy image Copy Code
ALTER TABLE Customer MERGE RANGE (10);

The new DDL for the table is:

Copy image Copy Code
CREATE TABLE Customer (
    id int NOT NULL,
    lastName varchar(20),
    orderCount int,
    orderDate date)
WITH
    ( DISTRIBUTION = HASH(id),
    PARTITION ( orderCount RANGE LEFT
    FOR VALUES (1, 5, 25, 50, 100)));

E. Splitting a partition

The following example splits a partition on a table.

The Customer table has the following DDL:

Copy image Copy Code
DROP TABLE Customer;

CREATE TABLE Customer (
    id int NOT NULL,
    lastName varchar(20),
    orderCount int,
    orderDate date)
WITH
    ( DISTRIBUTION = HASH(id),
    PARTITION ( orderCount RANGE LEFT
    FOR VALUES (1, 5, 10, 25, 50, 100 )));

The following command creates a new partition bound by the value 75, between 50 and 100.

Copy image Copy Code
ALTER TABLE Customer SPLIT RANGE (75);

The new DDL for the table is:

Copy image Copy Code
CREATE TABLE Customer (
   id int NOT NULL,
   lastName varchar(20),
   orderCount int,
   orderDate date)
   WITH DISTRIBUTION = HASH(id),
   PARTITION ( orderCount (RANGE LEFT
      FOR VALUES (1, 5, 10, 25, 50, 75, 100 )));

F. Using SWITCH to move a partition to a history table

The following example moves the data in a partition of the Orders table to a partition in the OrdersHistory table.

The Orders table has the following DDL:

Copy image Copy Code
CREATE TABLE Orders (
    id INT,
    city VARCHAR (25),
    lastUpdateDate DATE,
    orderDate DATE )
WITH
    (DISTRIBUTION = HASH ( id ),
    PARTITION ( orderDate RANGE RIGHT
    FOR VALUES ('2004-01-01', '2005-01-01', '2006-01-01', '2007-01-01' )));

In this example, the Orders table has the following partitions. Each partition contains data.

Partition | Has data? | Boundary range
1 | Yes | OrderDate < '2004-01-01'
2 | Yes | '2004-01-01' <= OrderDate < '2005-01-01'
3 | Yes | '2005-01-01' <= OrderDate< '2006-01-01'
4 | Yes | '2006-01-01'<= OrderDate < '2007-01-01'
5 | Yes | '2007-01-01' <= OrderDate

  • Partition 1 (has data): OrderDate < '2004-01-01'

  • Partition 2 (has data): '2004-01-01' <= OrderDate < '2005-01-01'

  • Partition 3 (has data): '2005-01-01' <= OrderDate< '2006-01-01'

  • Partition 4 (has data): '2006-01-01'<= OrderDate < '2007-01-01'

  • Partition 5 (has data): '2007-01-01' <= OrderDate

The OrdersHistory table has the following DDL, which has identical columns and column names as the Orders table. Both are hash-distributed on the id column.

Copy image Copy Code
CREATE TABLE OrdersHistory (
   id INT,
   city VARCHAR (25),
   lastUpdateDate DATE,
   orderDate DATE )
WITH
    (DISTRIBUTION = HASH ( id ),
    PARTITION ( orderDate RANGE RIGHT
    FOR VALUES ( '2004-01-01' )));

Although the columns and column names must be the same, the partition boundaries do not need to be the same. In this example, the OrdersHistory table has the following two partitions and both partitions are empty:

  • Partition 1 (no data): OrderDate < '2004-01-01'

  • Partition 2 (empty): '2004-01-01' <= OrderDate

For the previous two tables, the following command moves all rows with OrderDate < '2004-01-01' from the Orders table to the OrdersHistory table.

Copy image Copy Code
ALTER TABLE Orders SWITCH PARTITION 1 TO OrdersHistory PARTITION 1;

As a result, the first partition in Orders is empty and the first partition in OrdersHistory contains data. The tables now appear as follows:

Orders table

  • Partition 1 (empty): OrderDate < '2004-01-01'

  • Partition 2 (has data): '2004-01-01' <= OrderDate < '2005-01-01'

  • Partition 3 (has data): '2005-01-01' <= OrderDate< '2006-01-01'

  • Partition 4 (has data): '2006-01-01'<= OrderDate < '2007-01-01'

  • Partition 5 (has data): '2007-01-01' <= OrderDate

OrdersHistory table

  • Partition 1 (has data): OrderDate < '2004-01-01'

  • Partition 2 (empty): '2004-01-01' <= OrderDate

To clean up the Orders table, you can remove the empty partition by merging partitions 1 and 2 as follows:

Copy image Copy Code
ALTER TABLE Orders MERGE RANGE ('2004-01-01');

After the merge, the Orders table has the following partitions:

Orders table

  • Partition 1 (has data): OrderDate < '2005-01-01'

  • Partition 2 (has data): '2005-01-01' <= OrderDate< '2006-01-01'

  • Partition 3 (has data): '2006-01-01'<= OrderDate < '2007-01-01'

  • Partition 4 (has data): '2007-01-01' <= OrderDate

Suppose another year passes and you are ready to archive the year 2005. You can allocate an empty partition for the year 2005 in the OrdersHistory table by splitting the empty partition as follows:

Copy image Copy Code
ALTER TABLE OrdersHistory SPLIT RANGE ('2005-01-01');

After the split, the OrdersHistory table has the following partitions:

OrdersHistory table

  • Partition 1 (has data): OrderDate < '2004-01-01'

  • Partition 2 (empty): '2004-01-01' < '2005-01-01'

  • Partition 3 (empty): '2005-01-01' <= OrderDate

G. Using partition switching to update one partition at a time in a large table

If you need to update a value in a large table and think it will take too long to update the entire table, you can use partition switching to update one partition at a time. Follow these steps:

  1. Discover the table structure from the metadata.

  2. Create an empty table with the same structure.

  3. Switch a partition into the empty table, perform the update, and then switch the partition back to the full table. Repeat this step for each partition that needs to be updated.

H. Change the collation of a column

This example uses ALTER TABLE to change the collation of the LastName column in the dimEmployee table.

This change affects only the metadata.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
ALTER TABLE AdventureWorksPDW2012.dbo.dimEmployee
    ALTER COLUMN LastName nvarchar(50)
    COLLATE Latin1_General_100_CI_AS_KS_WS;

See Also


BACKUP DATABASE (SQL Server PDW)

[image]

BACKUP DATABASE (SQL Server PDW)
See Also Send Feedback

Creates a backup of a SQL Server PDW database and stores the backup off the appliance in a user-specified network location. Use this statement with RESTORE DATABASE (SQL Server PDW) for disaster recovery, or to copy a database from one appliance to another.

Before you begin, see Acquire and Configure a Backup Server (SQL Server PDW).

There are two types of backups in SQL Server PDW. A full database backup is a backup of an entire SQL Server PDW database. A differential database backup only includes changes made since the last full backup. A backup of a user database includes database users, and database roles. A backup of the master database includes logins.

For more information about SQL Server PDW database backups, see Backup and Restore (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

Create a full backup of a user database or the master database.
BACKUP DATABASE database_name
    TO DISK = '\\UNC_path\backup_directory'
    [ WITH [ ( ]  <with_options> [ ,...n ]  [ ) ] ]
[;]

Create a differential backup of a user database.
BACKUP DATABASE database_name
    TO DISK = '\\UNC_path\backup_directory'
    WITH [ ( ] DIFFERENTIAL
    [ , <with_options> [ ,...n ] [ ) ]
[;]

<with_options> ::=
    DESCRIPTION = 'text'
    | NAME = 'backup_name'

Arguments

database_name

The name of the database on which to create a backup. The database can be the master database or a user database.

TO DISK = '\\UNC_path\backup_directory'

The network path and directory to which SQL Server PDW will write the backup files. For example, '\\xxx.xxx.xxx.xxx\backups\2012\Monthly\08.2012.Mybackup'.

  • The path to the backup directory name must already exist and must be specified as a fully qualified universal naming convention (UNC) path.

  • The backup directory, backup_directory, must not exist before running the backup command. SQL Server PDW will create the backup directory.

  • The path to the backup directory cannot be a local path and it cannot be a location on any of the SQL Server PDW appliance nodes.

  • The maximum length of the UNC path and backup directory name is 200 characters.

  • The server or host must be specified as an IP address. You cannot specify it as the host or server name.

DESCRIPTION = 'text'

Specifies a textual description of the backup. The maximum length of the text is 255 characters.

The description is stored in the metadata, and will be displayed when the backup header is restored with RESTORE HEADERONLY.

NAME = 'backup _name'

Specifies the name of the backup. The backup name can be different from the database name.

  • Names can have a maximum of 128 characters.

  • Cannot include a path.

  • Must begin with a letter or number character or an underscore (_). Special characters permitted are the underscore (_), hyphen (-), or space ( ). Backup names cannot end with a space character.

  • The statement will fail if backup_name already exists in the specified location.

This name is stored in the metadata, and will be displayed when the backup header is restored with RESTORE HEADERONLY.

DIFFERENTIAL

Specifies to perform a differential backup of a user database. If omitted, the default is a full database backup. The name of the differential backup does not need to match the name of the full backup. For keeping track of the differential and its corresponding full backup, consider using the same name with 'full' or 'diff' appended.

For example:

BACKUP DATABASE Customer TO DISK = '\\xxx.xxx.xxx.xxx\backups\CustomerFull';

BACKUP DATABASE Customer TO DISK = '\\xxx.xxx.xxx.xxx\backups\CustomerDiff' WITH DIFFERENTIAL;

Permissions

Requires the BACKUP DATABASE permission or membership in the db_backupoperator fixed database role.

Requires a Windows account that has permission to access, create, and write to the backup directory. You must also store the Windows account name and password in SQL Server PDW. To add these network credentials to SQL Server PDW, use the sp_pdw_add_network_credentials (SQL Server PDW) stored procedure.

For more information about managing credentials in SQL Server PDW, see the Security section.

Error Handling

BACKUP DATABASE errors under the following conditions:

  • User permissions are not sufficient to perform a backup.

  • SQL Server PDW does not have the correct permissions to the network location where the backup will be stored.

  • The database does not exist.

  • The target directory already exists on the network share.

  • The target network share is not available.

  • The target network share does not have enough space for the backup. The BACKUP DATABASE command does not confirm that sufficient disk space exists prior to initiating the backup, making it possible to generate an out-of-disk-space error while running BACKUP DATABASE. When insufficient disk space occurs, SQL Server PDW rolls back the BACKUP DATABASE command.

  • Attempt to start a backup within a transaction.

General Remarks

A SQL Server PDW backup is stored as a set of multiple files within the same directory.

A differential backup usually takes less time than a full backup and can be performed more frequently. When multiple differential backups are based on the same full backup, each differential includes all of the changes in the previous differential backup.

If you cancel a BACKUP command, SQL Server PDW will remove the target directory and any files created for the backup. If SQL Server PDW loses network connectivity to the share, the rollback cannot complete.

Full backups and differential backups are stored in separate directories. Naming conventions are not enforced for specifying that a full backup and differential backup belong together. You can track this through your own naming conventions. Alternatively, you can track this by using the WITH DESCRIPTION option to add a description, and then by using the RESTORE HEADERONLY statement to retrieve the description.

Limitations and Restrictions

You cannot perform a differential backup of the master database. Only full backups of the master database are supported.

The backup files are stored in a format suitable only for restoring the backup to a SQL Server PDW appliance by using the RESTORE DATABASE (SQL Server PDW) statement.

The backup with the BACKUP DATABASE statement cannot be used to transfer data or user information to SMP SQL Server databases. For that functionality, you can use the remote table copy feature. For more information, see Remote Table Copy (SQL Server PDW).

SQL Server PDW uses SQL Server backup technology to backup and restore databases. SQL Server backup options are preconfigured to use backup compression. You cannot set backup options such as compression, checksum, block size, and buffer count.

Only one database backup or restore can run on the appliance at any given time. SQL Server PDW will queue backup or restore commands until the current backup or restore command has completed.

The target appliance for restoring the backup must have at least as many Compute nodes as the source appliance. The target can have more Compute nodes than the source appliance, but cannot have fewer Compute nodes.

SQL Server PDW does not track the location and names of backups since the backups are stored off the appliance.

SQL Server PDW does track the success or failure of database backups.

A differential backup is only allowed if the last full backup completed successfully. For example, suppose that on Monday you create a full backup of the Sales database and the backup finishes successfully. Then on Tuesday you create a full backup of the Sales database and it fails. After this failure, you cannot then create a differential backup based on Monday’s full backup. You must first create a successful full backup before creating a differential backup.

Metadata

These dynamic management views contain information about all backup, restore, and load operations. The information persists across system restarts.

  • sys.pdw_loader_backup_runs (SQL Server PDW)

  • sys.pdw_loader_backup_run_details (SQL Server PDW)

  • sys.pdw_loader_run_stages (SQL Server PDW)

Performance

To perform a backup, SQL Server PDW first backs up the metadata, and then it performs a parallel backup of the database data stored on the Compute nodes. Data is copied directly from each Compute nodes to the backup directory. To achieve the best performance for moving data from the Compute nodes to the backup directory, SQL Server PDW controls the number of Compute nodes that are copying data concurrently.

Locking

Takes an exclusive lock on the DATABASE object.

Security

SQL Server PDW backups are not stored on the appliance. Therefore, your IT team is responsible for managing all aspects of the backup security. For example, this includes managing the security of the backup data, the security of the server used to store backups, and the security of the networking infrastructure that connects the backup server to the SQL Server PDW appliance.

Manage Network Credentials

Network access to the backup directory is based on standard Windows file sharing security. Before performing a backup, you need to create or designate a Windows account that will be used for authenticating SQL Server PDW to the backup directory. This windows account must have permission to access, create, and write to the backup directory.

Important note Important

To reduce security risks with your data, we advise that you designate one Windows account solely for the purpose of performing backup and restore operations. Allow this account to have permissions to the backup location and nowhere else.

You need to store the user name and password in SQL Server PDW by running the sp_pdw_add_network_credentials (SQL Server PDW) stored procedure. SQL Server PDW uses Windows Credential Manager to store and encrypt user names and passwords on the Control node and Compute nodes. The credentials are not backed up with the BACKUP DATABASE command.

To remove network credentials from SQL Server PDW, see sp_pdw_remove_network_credentials (SQL Server PDW)

To list all of the network credentials stored in SQL Server PDW, use the sys.dm_pdw_network_credentials (SQL Server PDW) dynamic management view.

Examples

A. Add network credentials for the backup location

To create a backup, SQL Server PDW must have read/write permission to the backup directory. The following example shows how to add the credentials for a user. SQL Server PDW will store these credentials and use them to for backup and restore operations.

Important note Important

For security reasons, we recommend creating one domain account solely for the purpose of performing backups.

Copy image Copy Code
EXEC sp_pdw_add_network_credentials 'xxx.xxx.xxx.xxx', 'domain1\backupuser', '*****';

B. Remove network credentials for the backup location

The following example shows how to remove the credentials for a domain user from SQL Server PDW.

Copy image Copy Code
EXEC sp_pdw_remove_network_credentials 'xxx.xxx.xxx.xxx';

C. Create a full backup of a user database

The following example creates a full backup of the Invoices user database. SQL Server PDW will create the Invoices2013 directory and will save the backup files to the \\10.192.63.147\backups\yearly\Invoices2013Full directory.

Copy image Copy Code
BACKUP DATABASE Invoices TO DISK = '\\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Full';

D. Create a differential backup of a user database

The following example creates a differential backup, which includes all changes made since the last full backup of the Invoices database. SQL Server PDW will create the \\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Diff directory to which it will store the files. The description 'Invoices 2013 differential backup' will be stored with the header information for the backup.

The differential backup will only run successfully if the last full backup of Invoices completed successfully.

Copy image Copy Code
BACKUP DATABASE Invoices TO DISK = '\\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Diff'
    WITH DIFFERENTIAL,
    DESCRIPTION = 'Invoices 2013 differential backup';

E. Create a full backup of the master database

The following example creates a full backup of the master database and stores it in the directory '\\10.192.63.147\backups\2013\daily\20130722\master'.

Copy image Copy Code
BACKUP DATABASE master TO DISK = '\\xxx.xxx.xxx.xxx\backups\2013\daily\20130722\master';

F. Create a backup of appliance login information.

The master database stores the appliance login information. To backup the appliance login information you need to backup master.

The following example creates a full backup of the master database.

Copy image Copy Code
BACKUP DATABASE master TO DISK = '\\xxx.xxx.xxx.xxx\backups\2013\daily\20130722\master'
WITH (
    DESCRIPTION = 'Master Backup 20130722',
    NAME = 'login-backup'
)
;

See Also

Reference

RESTORE DATABASE (SQL Server PDW)


CREATE COLUMNSTORE INDEX (SQL Server PDW)

[image]

CREATE COLUMNSTORE INDEX (SQL Server PDW)
See Also Send Feedback

Creates or rebuilds an xVelocity memory-optimized clustered columnstore index on a SQL Server PDW table. Use this primarily to rebuild all partitions of a clustered columnstore index or to convert a small rowstore dimension table to a columnstore table.

To better understand clustered columnstore indexes, see Clustered Columnstore Indexes (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE CLUSTERED COLUMNSTORE INDEX index_name
    ON [ database_name . [ schema_name ] . | schema_name . ] table_name
    [ WITH ( DROP_EXISTING = { ON | OFF } ) ]
[;]

Arguments

index_name

Is the name of the clustered columnstore index. When DROP_EXISTING = ON is used, index_name is the name of an existing index.

Index names must follow the rules of identifiers.

database_name . [ schema_name ] . | schema_name . ] table_name

Specifies the one-, two-, or three-part name of the table that will have the index. The table can be a regular table or a temporary table.

DROP_EXISTING

Specifies whether to drop and rebuild and existing index, or to create a new index.

ON

Specifies to drop and then rebuild the existing index, specified by index_name , as a clustered columnstore index. The existing index can be a clustered index or a clustered columnstore index.

OFF

Specifies to create a new clustered columnstore index. All indexes must be dropped before using this option.

Permissions

Requires ALTER permission on the table, or membership in one of the following roles:

  • sysadmin fixed server role

  • db_ddladmin fixed database role

  • db_owner fixed database role

General Remarks

CREATE COLUMNSTORE INDEX with the DROP_EXISTING = ON option is the same as calling ALTER INDEX with the REBUILD PARTITION = ALL option. The difference between the two statements is that with ALTER INDEX you can specify one or more partitions, whereas with CREATE COLUMNSTORE INDEX you must rebuild the entire index.

For a distributed table, creating a clustered columnstore index applies after a table is created and the rows are distributed and partitioned. It does not affect the way the rows are distributed and partitioned.

For a replicated table, the clustered columnstore index applies after a table is created and replicated to the Compute nodes. It does not affect replication.

Limitations and Restrictions

You can create up to 16 clustered columnstore indexes concurrently on the appliance, as long as each index is created on a different table. You can queue additional requests for clustered columnstore indexes up to the SQL Server PDW request limit of 1024 total requests that can be running or waiting on the appliance.

Only one clustered columnstore index can exist on a table at any given time. If two or more clustered columnstore index requests are issued concurrently on the same table, they will block until the one that is currently running finishes. You can cancel the requests that are waiting, if desired.

Non-clustered indexes are not allowed on a table that has a clustered columnstore index.

Like other DDL statements in SQL Server PDW, you cannot use CREATE COLUMNSTORE INDEX within an explicit transaction. However, the PDW does guarantee the atomicity of each CREATE COLUMNSTORE INDEX statement. If a failure occurs while SQL Server PDW is creating a clustered columnstore index, all intermediate steps will be rolled back and the data will remain unchanged.

Locking Behavior

All read and write operations on the table are blocked while a clustered columnstore index is being created or rebuilt.

Metadata

When CREATE COLUMNSTORE INDEX converts a rowstore table is converted to a columnstore table, SQL Server PDW retains all metadata and objects associated with the rowstore table. For example, column definitions, default constraints, and permissions, are retained. Objects such as views and statistics on the table are retained.

All of the columns in a clustered columnstore index are stored in the metadata as included columns. The clustered columnstore index does not have key columns.

Related system catalog views:

  • sys.indexes (SQL Server PDW)

  • sys.index_columns (SQL Server PDW)

  • sys.partitions (SQL Server PDW)

Related dynamic management views (DMVs):

  • sys.pdw_nodes_column_store_segments (SQL Server PDW)

  • sys.pdw_nodes_column_store_dictionaries (SQL Server PDW)

  • sys.pdw_nodes_column_store_row_groups (SQL Server PDW)

Performance

CREATE COLUMNSTORE INDEX runs in parallel across the Compute nodes for both replicated and distributed tables, and runs serially across the distributions for each Compute node which is slow for large table. However, CREATE TABLE AS SELECT performs in parallel across the Compute nodes and in parallel across the distributions within each Compute node. This is one of the reasons why we recommend using CREATE TABLE AS SELECT instead of CREATE COLUMNSTORE INDEX for converting large tables from rowstore to columnstore.

Examples

A. Change a clustered index to a clustered columnstore index

By using the CREATE CLUSTERED COLUMNSTORE INDEX statement with DROP_EXISTING = ON, you can:

  • Change a clustered index into a clustered columnstore index.

  • Rebuild a clustered columnstore index.

This example creates the xDimProduct table as a rowstore table with a clustered index, and then uses CREATE CLUSTERED COLUMNSTORE INDEX to change the table from a rowstore table to a columnstore table.

Copy image Copy Code
USE AdventureWorksPDW2012;

IF EXISTS (SELECT name FROM sys.tables
    WHERE name = N'xDimProduct'
    AND object_id = OBJECT_ID (N'xDimProduct'))
DROP TABLE xDimProduct;

--Create a distributed table with a clustered index.
CREATE TABLE xDimProduct (ProductKey, ProductAlternateKey, ProductSubcategoryKey)
WITH ( DISTRIBUTION = HASH(ProductKey),
    CLUSTERED INDEX (ProductKey) )
AS SELECT ProductKey, ProductAlternateKey, ProductSubcategoryKey FROM DimProduct;

--Change the existing clustered index
--to a clustered columnstore index with the same name.
--Look up the name of the index before running this statement.
CREATE CLUSTERED COLUMNSTORE INDEX <index_name>
ON xdimProduct
WITH ( DROP_EXISTING = ON );

B. Rebuild a clustered columnstore index

Building on the previous example, this example uses CREATE CLUSTERED COLUMNSTORE INDEX to rebuild the existing clustered columnstore index called cci_xDimProduct.

Copy image Copy Code
--Rebuild the existing clustered columnstore index.
CREATE CLUSTERED COLUMNSTORE INDEX cci_xDimProduct
ON xdimProduct
WITH ( DROP_EXISTING = ON );

C. Change the name of a clustered columnstore index

To change the name of a clustered columnstore index, drop the existing clustered columnstore index, and then recreate the index with a new name.

We recommend only doing this operation with a small table or an empty table. It will take a long time to drop a large clustered columnstore index and rebuild with a different name.

Using the cci_xDimProduct clustered columnstore index from the previous example, this example drops the cci_xDimProduct clustered columnstore index and then recreates the clustered columnstore index with the name mycci_xDimProduct.

Copy image Copy Code
--For illustration purposes, drop the clustered columnstore index.
--The table continues to be distributed, but changes to a heap.
DROP INDEX cci_xdimProduct ON xDimProduct;

--Create a clustered index with a new name, mycci_xDimProduct.
CREATE CLUSTERED COLUMNSTORE INDEX mycci_xDimProduct
ON xdimProduct
WITH ( DROP_EXISTING = OFF );

D. Convert a columnstore table to a rowstore table with a clustered index

There might be a situation for which you want to drop a clustered columnstore index and create a clustered index. This stores the table in rowstore format. This example converts a columnstore table to a rowstore table with a clustered index with the same name. None of the data is lost. All data goes to the rowstore table and the columns listed become the key columns in the clustered index.

Copy image Copy Code
--Drop the clustered columnstore index and create a clustered rowstore index.
--All of the columns will be stored in the rowstore clustered index.
--The columns listed will be the included columns in the index.
CREATE CLUSTERED INDEX cci_xDimProduct
ON xdimProduct (ProductKey, ProductAlternateKey, ProductSubcategoryKey, WeightUnitMeasureCode)
WITH ( DROP_EXISTING = ON);

E. Convert a columnstore table back to a rowstore heap

Use DROP INDEX (SQL Server PDW) to drop the clustered columnstore index and convert the table to a rowstore heap. This example converts the cci_xDimProduct table to a rowstore heap. The table continues to be distributed, but is stored as a heap.

Copy image Copy Code
--Drop the clustered columnstore index. The table continues to be distributed, but changes to a heap.
DROP INDEX cci_xdimProduct ON xdimProduct;

See Also


CREATE DATABASE (SQL Server PDW)

[image]

CREATE DATABASE (SQL Server PDW)
See Also Send Feedback

Creates a new database on a SQL Server PDW appliance. Use this statement to create all files associated with an appliance database and to set maximum size and auto-growth options for the database tables and transaction log.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE DATABASE database_name
WITH (
    [ AUTOGROW = ON | OFF , ]
    REPLICATED_SIZE = replicated_size [ GB ] ,
    DISTRIBUTED_SIZE = distributed_size [ GB ] ,
    LOG_SIZE = log_size [ GB ] )
[;]

Arguments

database_name

The name of the new database. For more information on permitted database names, see Object Naming Rules (SQL Server PDW). You cannot name a database any of these Reserved Database Names (SQL Server PDW).

AUTOGROW = ON | OFF

Specifies whether the replicated_size, distributed_size, and log_size parameters for this database will automatically grow as needed beyond their specified sizes. Default value is OFF.

If AUTOGROW is ON, replicated_size, distributed_size, and log_size will grow as required (not in blocks of the initial specified size) with each data insert, update, or other action that requires more storage than has already been allocated.

If AUTOGROW is OFF, the sizes will not grow automatically. SQL Server PDW will return an error when attempting an action that requires replicated_size, distributed_size, or log_size to grow beyond their specified value.

AUTOGROW is either ON for all sizes or OFF for all sizes. For example, it is not possible to set AUTOGROW ON for log_size, but not set it for replicated_size.

replicated_size [ GB ]

A positive number. Sets the size (in integer or decimal gigabytes) for the total space allocated to replicated tables and corresponding data on each Compute node. For minimum and maximum replicated_size requirements, see Minimum and Maximum Values (SQL Server PDW).

If AUTOGROW is ON, replicated tables will be permitted to grow beyond this limit.

If AUTOGROW is OFF, an error will be returned if a user attempts to create a new replicated table, insert data into an existing replicated table, or update an existing replicated table in a manner that would increase the size beyond replicated_size.

distributed_size [ GB ]

A positive number. The size, in integer or decimal gigabytes, for the total space allocated to distributed tables (and corresponding data) across the appliance. For minimum and maximum distributed_size requirements, see Minimum and Maximum Values (SQL Server PDW).

If AUTOGROW is ON, distributed tables will be permitted to grow beyond this limit.

If AUTOGROW is OFF, an error will be returned if a user attempts to create a new distributed table, insert data into an existing distributed table, or update an existing distributed table in a manner that would increase the size beyond distributed_size.

log_size [ GB ]

A positive number. The size (in integer or decimal gigabytes) for the transaction log across the appliance.

For minimum and maximum log_size requirements, see Minimum and Maximum Values (SQL Server PDW).

If AUTOGROW is ON, the log file is permitted to grow beyond this limit. Use the DBCC SHRINKLOG (SQL Server PDW) statement to reduce the size of the log files to their original size.

If AUTOGROW is OFF, an error will be returned to the user for any action that would increase the log size on an individual Compute node beyond log_size.

Permissions

Requires the CREATE ANY DATABASE permission in the master database, or membership in the sysadmin fixed server role.

The following example provides the permission to create a database to the database user Fay.

Copy image Copy Code
USE master;
GO
GRANT CREATE ANY DATABASE TO [Fay];
GO

General Remarks

Databases are created with database compatibility level 120, which is the compatibility level for SQL Server 2014. This ensures the database will be able to use all of the SQL Server 2014 functionality that PDW uses.

Limitations and Restrictions

The CREATE DATABASE statement is not allowed in an explicit transaction. For more information, see Transactions (SQL Server PDW).

For information on minimum and maximum constraints on databases, see Minimum and Maximum Values (SQL Server PDW).

At the time a database is created, there must be enough available free space on each Compute node to allocate the combined total of the following sizes:

  • SQL Server database with tables the size of replicated_table_size.

  • SQL Server database with tables the size of (distributed_table_size / number of Compute nodes ).

  • SQL Server logs the size of (log_size / number of Compute nodes.

Locking

Takes a shared lock on the DATABASE object.

Metadata

After this operation succeeds, an entry for this database will appear in the sys.databases (SQL Server PDW) and sys.objects (SQL Server PDW)metadata views.

Examples

A. Basic database creation examples

The following example creates the database mytest with a storage allocation of 100 GB per Compute node for replicated tables, 500 GB per appliance for distributed tables, and 100 GB per appliance for the transaction log. In this example, AUTOGROW is off by default.

Copy image Copy Code
CREATE DATABASE mytest
   WITH
   (REPLICATED_SIZE = 100 GB,
   DISTRIBUTED_SIZE = 500 GB,
   LOG_SIZE = 100 GB );

The following example creates the database mytest with the same parameters as above, except that AUTOGROW is turned on. This allows the database to grow outside the specified size parameters.

Copy image Copy Code
CREATE DATABASE mytest
   WITH
   (AUTOGROW = ON,
   REPLICATED_SIZE = 100 GB,
   DISTRIBUTED_SIZE = 500 GB,
   LOG_SIZE = 100 GB);

B. Creating a database with partial gigabyte sizes

The following example creates the database mytest, with AUTOGROW off, a storage allocation of 1.5 GB per Compute node for replicated tables, 5.25 GB per appliance for distributed tables, and 10 GB per appliance for the transaction log.

Copy image Copy Code
CREATE DATABASE mytest
   WITH
   (REPLICATED_SIZE = 1.5 GB,
   DISTRIBUTED_SIZE = 5.25 GB,
   LOG_SIZE = 10 GB);

See Also

Reference

ALTER DATABASE (SQL Server PDW)
DROP DATABASE (SQL Server PDW)


CREATE EXTERNAL DATA SOURCE (SQL Server PDW)

[image]

CREATE EXTERNAL DATA SOURCE (SQL Server PDW)
See Also Send Feedback

Creates a SQL Server PDW external data source for data stored in hdfs or Azure blob storage (WASB). Use this for PolyBase scenarios that integrate PDW with Hadoop or Azure blob storage (WASB).

To use PolyBase integration, you need to create an external data source, an external file format, and an external table. For more information, see PolyBase for Hadoop and WASB Integration (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

--Create an external data source for a Hadoop cluster
CREATE EXTERNAL DATA SOURCE data_source_name
    WITH (
        TYPE = HADOOP,
        LOCATION = 'hdfs://NameNode_URI[:port]'
        [, JOB_TRACKER_LOCATION = 'JobTracker_URI[:port]' ]
    )
[;]

--Create an external data source for Windows Azure storage blob
CREATE EXTERNAL DATA SOURCE data_source_name
    WITH (
        TYPE = HADOOP,
        LOCATION = 'wasb[s]://[ container@ ] account_name.blob.core.windows.net/path'
    )
[;]

Arguments

data_source_name

Specifies the user-defined name for the data source. The name must be unique for SQL Server PDW.

TYPE = HADOOP

Specifies the data source type. HADOOP is the only supported type in this release.

LOCATION = 'hdfs://NameNode_URI[:port]'

Specifies the Uniform Resource Indicator (URI) for a Hadoop external data source.

Important note Important

The LOCATION value is a string and is not validated when you create the external data source. Entering an incorrect value can cause future delays when accessing the location. To avoid typing errors, we recommend to cut and paste the appropriate example and then replace the italicized parameters with the correct ones for your Hadoop cluster.

Examples:

  • HDInsight on APS:

    Copy this code and replace F12345 with your appliance domain. The –C specifies to use the InfiniBand network, and HHN01 is the name of the HDInsight Head node; these are constant and must be present.

    Copy image Copy Code
    LOCATION = 'hdfs://F12345-C-HHN01:8020'

  • Hortonworks or Cloudera

    Copy this code and replace NameNode_IP with the IP address of your Name node.

    Copy image Copy Code
    LOCATION = 'hdfs://NameNode_IP:8020'
JOB_TRACKER_LOCATION = 'JobTracker_URI[:port]'

Specifies to push predicate computation to the Hadoop cluster. The port number for JOB_TRACKER_LOCATION cannot be the same as the port number for LOCATION. It is specific to the Hadoop distribution.

Important note Important

For Hortonworks on HDP 2.0, JOB_TRACKER_LOCATION port is not optional. For HDP 2.0 on Windows, the port must be 8032. For HDP 2.0 on Linux, the port must be 8050.

Important note Important

The JOB_TRACKER_LOCATION value is a string and is not validated when you create the external data source. Entering an incorrect value can cause future delays when accessing the location. To avoid typing errors, we recommend to cut and paste the appropriate example and then replace the italicized parameters with the correct ones for your Hadoop cluster.

Examples:

In these examples, replace F12345 with your appliance domain_name, and replace NameNode_IP with the IP address of your Name node.

  • HDI on APS with HDP 2.0:

    Copy image Copy Code
    JOB_TRACKER_LOCATION = 'F12345-C-HHN01:8032'

  • HDI on APS with HDP 1.3:

    Copy image Copy Code
    JOB_TRACKER_LOCATION = 'F12345-C-HHN01:50300'

  • Hortonworks HDP 2.0 on Windows:

    Copy image Copy Code
    JOB_TRACKER_LOCATION = 'hdfs://NameNode_IP:8032'

  • Hortonworks HDP 1.3 on Windows:

    Copy image Copy Code
    JOB_TRACKER_LOCATION = 'hdfs://NameNode_IP:50300'

  • Hortonworks HDP 2.0 on Linux:

    Copy image Copy Code
    JOB_TRACKER_LOCATION = 'hdfs://NameNode_IP:8050'

  • Hortonworks HDP 1.3 on Linux:

    Copy image Copy Code
    JOB_TRACKER_LOCATION = 'hdfs://NameNode_IP:50300'

  • Cloudera 4.3 on Linux:

    Copy image Copy Code
    JOB_TRACKER_LOCATION = 'hdfs://NameNode_IP:8021'
LOCATION = 'wasb[s]://[ container@ ]account_name.blob.core.windows.net/path'

Specifies the URI for connecting to Windows Azure blob storage.

wasb[s]

Specifies the protocol for Windows Azure blob storage. The [s] is optional and specifies a secure connection. We recommend using 'wasbs' instead of 'wasb'.

Note Note

The location can use asv[s] instead of wasb[s]. The asv[s] syntax is deprecated and will be removed in a future release.

container

Specifies the name of the Azure blob storage container. To specify the root container of a domain’s storage account, use the domain name instead of the container name. Root containers are read-only, so data cannot be written back to the container.

account_name

The fully qualified domain name (FQDN) of the Azure storage account. Default is the default file system.

path

Specifies the HDFS path to the blob storage file. The path looks like a directory path and uses / as a directory separator. The path is actually stored as a key and value pair instead of a hierarchical path. A blob’s key-value pair of day1/log1.txt specifies the key as day1 and the file as log1.txt.

Example:

Copy image Copy Code
LOCATION = 'wasbs://dailylogs@myaccount.blob.core.windows.net/'

Permissions

Requires CONTROL SERVER or ALTER ANY EXTERNAL DATA SOURCE permission.

Error Handling

A runtime error will occur if the external Hadoop data sources are inconsistent about having JOB_TRACKER_LOCATION defined.

SQL Server PDW does not verify the existence of the external data source when it creates the external data source object. If the data source does not exist during query execution, an error will occur.

General Remarks

The external data source is stored in master as a server level object.

When JOB_TRACKER_LOCATION is defined, SQL Server PDW will consider optimizing each query against the external data source by initiating a map reduce job on the Hadoop source. The results of the map reduce job will be transferred, in parallel, back to SQL Server PDW so that SQL Server PDW can continue to run the query.

Limitations and Restrictions

For HDP 2.0 clusters, when you specify JOB_TRACKER_LOCATION you must also specify the port. optional.

All data sources defined on the same Hadoop cluster location must use the same setting for JOB_TRACKER_LOCATION. If there is inconsistency, a runtime error will occur.

For HDFS connections, if the Hadoop cluster is setup with a name and the external data source uses the IP address for the cluster location, SQL Server PDW must still be able to resolve the cluster name when the data source is used. To enable PDW to resolve the name, you can use a DNS forwarder. For more information, see Use a DNS Forwarder to Resolve Non-Appliance DNS Names (Analytics Platform System).

Locking

Takes a shared lock on the EXTERNALDATASOURCE object.

Examples

A. Create an HDInsight on APS external data source

This example creates an external data source for the HDI Region on APS. LOCATION is the appliance domain name, followed by '-C-', and HHN01 which is the name of the HDI name node.

Caution note Caution

We recommend that you copy and paste this code and then replace F12345 with your appliance domain name. Specification mistakes in this code can cause significant delays.

Copy image Copy Code
CREATE EXTERNAL DATA SOURCE HDI_eds
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://F12345-C-HHN01:8020',

);

B. Create an HDI on APS external data source with predicate pushdown

This example creates an external data source for the HDI on APS. It is the same as the previous example, except that it uses predicate pushdown. When a PolyBase query selects from this external data source, some or all of the query predicate can be computed on Hadoop, which could reduce data transfer from Hadoop to PDW, and improve query performance .

Caution note Caution

We recommend that you copy and paste this code and then replace F12345 with your appliance domain name. Specification mistakes in this code can cause significant delays.

Copy image Copy Code
--For HDI Regions running Hortonworks HDP 2.0 on Windows
CREATE EXTERNAL DATA SOURCE HDI_pp_eds
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://F12345-C-HHN01:8020',
    JOB_TRACKER_LOCATION = 'F12345-C-HHN01:8032'
);

--For HDI Regions running Hortonworks HDP 1.3
CREATE EXTERNAL DATA SOURCE HDI_pp_eds
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://F12345-C-HHN01:8020',
    JOB_TRACKER_LOCATION = 'F12345-C-HHN01:50300'
);

C. Create a Hortonworks or Cloudera external data source

To create a Hortonworks or Cloudera external data source, specify the IP address of the Hadoop name node.In this example, replace 10.10.10.10 with the IP address of your Hadoop name node, and then specify port 8020 which is the default port.

Copy image Copy Code
--Hortonworks or Cloudera external data source
CREATE EXTERNAL DATA SOURCE hadoop_eds
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://10.10.10.10:8020'
);

D. Create a Hortonworks or Cloudera external data source with predicate pushdown

These examples use the JOB_TRACKER_LOCATION option to specify that PolyBase queries will perform predicate pushdown when the query optimizer deems it to be advantageous.

Each Hadoop distribution uses a different port for the job tracker location. These examples give the default port for each supported Hadoop distribution.

In the following examples, the JOB_TRACKER_LOCATION uses ports 8050 and 50300 which are the job tracker ports for Hortonworks 2.0 on Linux and 1.3 on Windows or Linux respectively.

Copy image Copy Code
--Create an external data source to Hortonworks HDP 2.0 on Linux
--and enable predicate pushdown.
CREATE EXTERNAL DATA SOURCE Hortonworks_pp_eds
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://10.10.10.10:8020',
    JOB_TRACKER_LOCATION = '10.10.10.10:8050'
);

--Create an external data source to Hortonworks HDP 2.0 on Windows
--and enable predicate pushdown.
CREATE EXTERNAL DATA SOURCE Hortonworks_pp_eds
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://10.10.10.10:8020',
    JOB_TRACKER_LOCATION = '10.10.10.10:8032'
);

--Create an external data source to Hortonworks HDP 1.3 on Windows or Linux
--and enable predicate pushdown.
CREATE EXTERNAL DATA SOURCE Hortonworks_pp_eds
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://10.10.10.10:8020',
    JOB_TRACKER_LOCATION = '10.10.10.10:50300'
);

In this next example, the JOB_TRACKER_LOCATION uses port 8021 which is the default port for Cloudera 4.3.

Copy image Copy Code
--Create an external data source to Cloudera 4.3 on Linux
--and enable predicate pushdown.
CREATE EXTERNAL DATA SOURCE Cloudera_pp_eds
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://10.10.10.10:8020',
    JOB_TRACKER_LOCATION = 10.10.10.10:8021'
);

D. Create a Windows Azure blob storage external data source

In this example, the data source is a Windows Azure storage blob called dailylogs in the Windows Azure account named myaccount. The WASB data source is for data transfer only; it does not support predicate pushdown.

Copy image Copy Code
CREATE EXTERNAL DATA SOURCE wasb1
WITH (
    TYPE = HADOOP,
    LOCATION = 'wasbs://dailylogs@myaccount.blob.core.windows.net/'
);

See Also

Reference

CREATE EXTERNAL TABLE (SQL Server PDW)
CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW)

Concepts

PolyBase for Hadoop and WASB Integration (SQL Server PDW)


CREATE EXTERNAL FILE FORMAT (SQL Server PDW)

[image]

CREATE EXTERNAL FILE FORMAT (SQL Server PDW)
See Also Send Feedback

Creates a SQL Server PDW external file format definition for external data stored in Hadoop or Azure blob storage. This is a prerequisite for creating an external table.

To create an external table, see CREATE EXTERNAL TABLE (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

--Create an external file format for a Hadoop ORC File.
CREATE EXTERNAL FILE FORMAT file_format_name
WITH (
    FORMAT_TYPE = ORC
    [ , DATA_COMPRESSION = {
        'org.apache.hadoop.io.compress.SnappyCodec'
      | 'org.apache.hadoop.io.compress.DefaultCodec'
      }
    ]
);

--Create an external file format for a Hadoop RcFile.
CREATE EXTERNAL FILE FORMAT file_format_name
WITH (
    FORMAT_TYPE = RCFILE,
    SERDE_METHOD = {
        'org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe'
      | 'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe'
    }
    [ , DATA_COMPRESSION = 'org.apache.hadoop.io.compress.DefaultCodec' ]
);

--Create an external file format for a Hadoop text-delimited file.
CREATE EXTERNAL FILE FORMAT file_format_name
WITH (
    FORMAT_TYPE = DELIMITEDTEXT
    [ , FORMAT_OPTIONS ( <format_options> [ ,...n ] ) ]
    [ , DATA_COMPRESSION = {
           'org.apache.hadoop.io.compress.GzipCodec'
         | 'org.apache.hadoop.io.compress.DefaultCodec'
        }
    ]

);

<format_options> ::=
{
    FIELD_TERMINATOR = field_terminator
    | STRING_DELIMITER = string_delimiter
    | DATE_FORMAT = datetime_format
    | USE_TYPE_DEFAULT = { TRUE | FALSE }
}

Arguments

file_format_name

Specifies a name for the new external file format.

ORC

Specifies Optimized Row Columnar for the external file format. This option requires Hive version 0.11 or higher. In conjunction with HDP 2.0, ORC file formats offer better compression and performance than the RCFILE formats.

RCFILE, SERDE_METHOD = SERDE_method

Specifies Record Columnar File (RcFile) for the external file format. This option requires a Hive Serializer and Deserializer (SerDe) method. The SerDe method is case sensitive.

Examples:

  • FORMAT_TYPE = RCFILE, SERDE_METHOD = 'org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe'

  • FORMAT_TYPE = RCFILE, SERDE_METHOD = 'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe'

DELIMITEDTEXT

Specifies the external file format contains text with delimiters.

FIELD_TERMINATOR = field_terminator

The terminator for each field (column) in the text-delimited file on Hadoop. The field_terminator can be can be one or more characters. The default is the pipe character ꞌ|ꞌ.

Examples:

  • FIELD_TERMINATOR = '|'

  • FIELD_TERMINATOR = ' '

  • FIELD_TERMINATOR = ꞌ\tꞌ

  • FIELD_TERMINATOR = '~|~'

STRING_DELIMITER = string_delimiter

Specifies the delimiter for data of type string in the text-delimited file on Hadoop. The string delimiter is one or more characters in length and is enclosed with single quotes. The default is the empty string "".

Examples:

  • STRING_DELIMITER = '*'

  • STRING_DELIMITER = ꞌ,ꞌ

  • STRING_DELIMITER = '0x7E0x7E'

DATE_FORMAT = datetime_format

Specifies a single format for all date and time data. If DATE_FORMAT is not specified or is the empty string, PolyBase will use the default format.

Default date formats:

  • Date: 'yyyy-MM-dd'

  • DateTime: 'yyyy-MM-dd HH:mm:ss'

  • DateTime2: 'yyyy-MM-dd HH:mm:ss'

  • DateTimeOffset: 'yyyy-MM-dd HH:mm:ss'

  • SmallDateTime: 'yyyy-MM-dd HH:mm'

  • Time: 'HH:mm:ss'

Supported date formats.

Format

datetime

smalldatetime

date

datetime2

datetimeoffset

mdy

[M[M]]M-[d]d-[yy]yy HH:mm:ss[.nnn]

[M[M]]M-[d]d-[yy]yy HH:mm[:00]

[M[M]]M-[d]d-[yy]yy

[M[M]]M-[d]d-[yy]yy HH:mm:ss[.nnnnnnn]

[M[M]]M-[d]d-[yy]yy HH:mm:ss[.nnnnnnn] zzz

[M[M]]M-[d]d-[yy]yy hh:mm:ss[.nnn][tt]

[M[M]]M-[d]d-[yy]yy hh:mm[:00][tt]

[M[M]]M-[d]d-[yy]yy hh:mm:ss[.nnnnnnn][tt]

[M[M]]M-[d]d-[yy]yy hh:mm:ss[.nnnnnnn][tt] zzz

myd

[M[M]]M-[yy]yy-[d]d HH:mm:ss[.nnn]

[M[M]]M-[yy]yy-[d]d HH:mm[:00]

[M[M]]M-[yy]yy-[d]d

[M[M]]M-[yy]yy-[d]d HH:mm:ss[.nnnnnnn]

[M[M]]M-[yy]yy-[d]d HH:mm:ss[.nnnnnnn] zzz

[M[M]]M-[yy]yy-[d]d hh:mm:ss[.nnn][tt]

[M[M]]M-[yy]yy-[d]d hh:mm[:00][tt]

[M[M]]M-[yy]yy-[d]d hh:mm:ss[.nnnnnnn][tt]

[M[M]]M-[yy]yy-[d]d hh:mm:ss[.nnnnnnn][tt] zzz

ymd

[yy]yy-[M[M]]M-[d]d HH:mm:ss[.nnn]

[yy]yy-[M[M]]M-[d]d HH:mm[:00]

[yy]yy-[M[M]]M-[d]d

[yy]yy-[M[M]]M-[d]d HH:mm:ss[.nnnnnnn]

[yy]yy-[M[M]]M-[d]d HH:mm:ss[.nnnnnnn] zzz

[yy]yy-[M[M]]M-[d]d hh:mm:ss[.nnn][tt]

[yy]yy-[M[M]]M-[d]d hh:mm[:00][tt]

[yy]yy-[M[M]]M-[d]d hh:mm:ss[.nnnnnnn][tt]

[yy]yy-[M[M]]M-[d]d hh:mm:ss[.nnnnnnn][tt] zzz

ydm

[yy]yy-[d]d-[M[M]]M HH:mm:ss[.nnn]

[yy]yy-[d]d-[M[M]]M HH:mm[:00]

[yy]yy-[d]d-[M[M]]M

[yy]yy-[d]d-[M[M]]M HH:mm:ss[.nnnnnnn]

[yy]yy-[d]d-[M[M]]M HH:mm:ss[.nnnnnnn] zzz

[yy]yy-[d]d-[M[M]]M hh:mm:ss[.nnn][tt]

[yy]yy-[d]d-[M[M]]M hh:mm[:00][tt]

[yy]yy-[d]d-[M[M]]M hh:mm:ss[.nnnnnnn][tt]

[yy]yy-[d]d-[M[M]]M hh:mm:ss[.nnnnnnn][tt] zzz

dmy

[d]d-[M[M]]M-[yy]yy HH:mm:ss[.nnn]

[d]d-[M[M]]M-[yy]yy HH:mm[:00]

[d]d-[M[M]]M-[yy]yy

[d]d-[M[M]]M-[yy]yy HH:mm:ss[.nnnnnnn]

[d]d-[M[M]]M-[yy]yy HH:mm:ss[.nnnnnnn] zzz

[d]d-[M[M]]M-[yy]yy hh:mm:ss[.nnn][tt]

[d]d-[M[M]]M-[yy]yy hh:mm[:00][tt]

[d]d-[M[M]]M-[yy]yy hh:mm:ss[.nnnnnnn][tt]

[d]d-[M[M]]M-[yy]yy hh:mm:ss[.nnnnnnn][tt] zzz

dym

[d]d-[yy]yy-[M[M]]M HH:mm:ss[.nnn]

[d]d-[yy]yy-[M[M]]M HH:mm[:00]

[d]d-[yy]yy-[M[M]]M

[d]d-[yy]yy-[M[M]]M HH:mm:ss[.nnnnnnn]

[d]d-[yy]yy-[M[M]]M HH:mm:ss[.nnnnnnn] zzz

[d]d-[yy]yy-[M[M]]M hh:mm:ss[.nnn][tt]

[d]d-[yy]yy-[M[M]]M hh:mm[:00][tt]

[d]d-[yy]yy-[M[M]]M hh:mm:ss[.nnnnnnn][tt]

[d]d-[yy]yy-[M[M]]M hh:mm:ss[.nnnnnnn][tt] zzz

Details:

  • To separate month, day and year values, you can use ' – ', ' / ', or ' . '. For simplicity, the table uses only the ' – ' separator.

  • To specify the month as text use three or more characters. Months with 1 or 2 characters will be interpreted as a number.

  • To separate time values, use the ' : ' symbol.

  • Letters enclosed in square brackets are optional.

  • The letters 'tt' designate [AM|PM|am|pm]. AM is the default. When 'tt' is specified, the hour value (hh) must be in the range of 0 to 12.

  • The letters 'zzz' designate the time zone offset for the system's current time zone in the format {+|-}HH:ss].

USE_TYPE_DEFAULT = { TRUE | FALSE }

Specifies how to handle missing values when SQL Server PDW is importing data from HDFS text files into SQL Server PDW, and when SQL Server PDW is exporting data from SQL Server PDW query results to the HDFS text file. No action occurs when CREATE EXTERNAL TABLE AS SELECT is run.

TRUE

For importing, all missing values in the HDFS delimited text file are stored using the column default in the external table definition. For example, a missing value in the HDFS text file imports to:

  • The default constraint if a default constraint is defined on the column.

  • 0 if the column is defined as a numeric column.

  • Empty string "" if the column is a string column.

FALSE

For importing, all missing values in the HDFS delimited text file are stored as NULL in the imported row data. Any NULL values that are stored by using the word NULL in the Hadoop file will be imported as the string 'NULL'.

For exporting, NULL column values in the SQL Server PDW data are stored as empty values in the HDFS text file. For example, if the field terminator is '|', a NULL value is stored by writing nothing between the field terminators: '||'.

DATA_COMPRESSION = data_compression_method

Specifies the data compression method for the data in the external file. When DATA_COMPRESSION is not specified, the default is uncompressed data.

The DELIMITEDTEXT format type supports these compression methods:

  • DATA COMPRESSION = 'org.apache.hadoop.io.compress.GzipCodec'

  • DATA COMPRESSION = 'org.apache.hadoop.io.compress.DefaultCodec'

The RCFILE format type supports this compression method:

  • DATA COMPRESSION = 'org.apache.hadoop.io.compress.DefaultCodec'

The ORC file format type supports these compression methods:

  • DATA COMPRESSION = 'org.apache.hadoop.io.compress.SnappyCodec'

  • DATA COMPRESSION = 'org.apache.hadoop.io.compress.DefaultCodec'

Metadata

The external file format is stored as a server object

Permissions

Requires CONTROL SERVER or ALTER ANY EXTERNAL FILE FORMAT permission.

General Remarks

Each external file format is stored in the master database as a server-level object.

For operations on compressed external file formats, PDW will perform the compression when the external table is created with CREATE EXTERNAL TABLE AS SELECT. PDW does not perform the compression when the external table is created with CREATE EXTERNAL TABLE. In all cases, PDW decompresses the data in order to return the query results. For predicate pushdown on external compressed delimited text files, Hadoop decompresses the data, performs the predicate calculation, and then compresses the results for transfer back to PDW.

Limitations and Restrictions

The row delimiter must be UTF-8 and supported by Hadoop’s LineRecordReader. In this release, the encoding is UTF-8 and the row delimiter is '\n'. These are not user-configurable.

Locking

Takes a shared lock on the EXTERNALFILEFORMAT object.

Performance

Recommendations for compressed external file formats

The impact of compressed files on performance is a tradeoff between transferring less data between the Hadoop Cluster and PDW, and increasing the CPU usage to compress and decompress the data. Usually, the PDW CPU’s are busy and compression will slow performance. Network bandwidth might be limited for transferring data back and forth to Azure storage blob over the Internet. In this case, compression could improve performance.

If you are using Hadoop to generate the data for a compressed external file, we recommend generating multiple files that are all stored in the same directory. Compressed files are decompressed as a serial operation. By reading multiple compressed files, PDW can spread the decompression across the Compute nodes. The ideal number of compressed files is proportionate to the number of Compute nodes multiplied by 8. This allows PDW to read and decompress, in parallel, one or more compressed files per distribution across all the Compute nodes.

Examples

Create a delimited text file format

This example creates an external file format named textdelimited1 for a text-delimited Hadoop file. The FORMAT_OPTIONS specify the fields in the Hadoop file will be separated with a pipe character '|'. The text file is also compressed with the Gzip codec. If DATA_COMPRESSION is not specified, the text file is uncompressed.

The data compression method can specify either the default Codec, 'org.apache.hadoop.io.compress.DefaultCodec', or the Gzip Codec, 'org.apache.hadoop.io.compress.GzipCodec'.

Copy image Copy Code
CREATE EXTERNAL FILE FORMAT textdelimited1
WITH (
    FORMAT_TYPE = DELIMITEDTEXT,
    FORMAT_OPTIONS (
        FIELD_TERMINATOR = '|',
        DATE_FORMAT = 'MM/dd/yyyy'
    ),
    DATA_COMPRESSION = 'org.apache.hadoop.io.compress.GzipCodec'
)
;

Create an RcFile external file format

This example creates a file format for a Hadoop RcFile that uses the serialization/deserialization method org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe. It also specifies to use the Default Codec for the data compression method. If DATA_COMPRESSION is not specified, the default is no compression.

Important note Important

The GZip Codec does not work with RcFiles, even though it compiles successfully.

Copy image Copy Code
CREATE EXTERNAL FILE FORMAT rcfile1
WITH (
    FORMAT_TYPE = RCFILE,
    SERDE_METHOD = 'org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe',
    DATA_COMPRESSION = 'org.apache.hadoop.io.compress.DefaultCodec'
)
;

Create an ORC external file format

This example creates a file format for a Hadoop ORC file that compresses the data with the org.apache.io.compress.SnappyCodec data compression method. If DATA_COMPRESSION is not specified, the default is no compression.

Copy image Copy Code
CREATE EXTERNAL FILE FORMAT textdelimited1
WITH (
    FORMAT_TYPE = ORC,
    DATA_COMPRESSION = 'org.apache.hadoop.io.compress.SnappyCodec'
)
;

See Also

Reference

CREATE EXTERNAL TABLE (SQL Server PDW)
CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW)

Concepts

PolyBase for Hadoop and WASB Integration (SQL Server PDW)


CREATE INDEX (SQL Server PDW)

[image]

CREATE INDEX (SQL Server PDW)
See Also Send Feedback

Creates a rowstore index on a table in SQL Server PDW. Use this statement to improve query performance.

A rowstore index stores and runs your table data by rows. Previously, this was the only way to store and run data in SQL Server PDW. Starting with this release, you can use a clustered columnstore index to run your data by columns. The clustered columnstore index gives tremendous benefits for both performance and data compression.

For more information on storing, accessing, and updating columnar data, see CREATE COLUMNSTORE INDEX (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE [ CLUSTERED | NONCLUSTERED ] INDEX index_name
    ON [ database_name . [ schema_name ] . | schema_name . ] table_name
        ( { column [ ASC | DESC ] } [ ,...n ] )
    WITH ( DROP_EXISTING = { ON | OFF } )
[;]

Arguments

CLUSTERED | NONCLUSTERED

Specifies whether the index is clustered or nonclustered.

CLUSTERED

Specifies to create an index in which the actual data is stored according to the sorted order of the column values. Each table can have only one clustered index at the same time.

To avoid rebuilding nonclustered indexes, create the clustered index first and then create any nonclustered indexes. All existing nonclustered indexes on a table are rebuilt when a clustered index is created.

For distributed tables, a clustered index affects the way data is stored within each distribution across the Compute nodes; it does not affect which rows are assigned to each distribution. For replicated tables, the clustered index affects the way the data is stored within each replicated table; it does not affect where the replicated tables are stored.

NONCLUSTERED

Specifies to create an index which stores and sorts the index values separately from the full table data. With a nonclustered index, the physical order of the actual data rows is independent of the indexed order.

index_name

The name of the index to be created. Index names must be unique per table. For more information on permitted index names, see Object Naming Rules (SQL Server PDW).

[ database_name . [ schema_name ] . | schema_name . ] table_name

The three-part name of the table to be indexed. The table name can optionally include the schema, or the database and schema.

The table can be empty or contain data. Either way, the index will be updated when data is added, deleted, or modified.

column_name [ ASC | DESC ] } [ ,...n ]

The list of key columns to include in the index. For each column, you can specify ascending or descending sort order. Default is ASC.

The key columns are combined to form a single composite index key and should be listed in sort-priority order. For the maximum number of key columns and the maximum allowable size of the combined index values, see Minimum and Maximum Values (SQL Server PDW).

DROP_EXISTING = { ON | OFF }

Is an option to rebuild an existing index with modified column specifications, and keep the same name for the index.

ON

Specifies to drop and rebuild the existing index, which must have the same name as the parameter index_name.

OFF

Specifies not to drop and rebuild the existing index. SQL Server PDW will display an error if the specified index name already exists.

With DROP_EXISTING, you can change:

  • A nonclustered rowstore index to a clustered rowstore index.

With DROP_EXISTING, you cannot change:

  • A clustered rowstore index to a nonclustered rowstore index.

  • A clustered columnstore index to any type of rowstore index.

Permissions

Requires ALTER permission on the table or membership in the db_ddladmin fixed database role.

General Remarks

SQL Server PDW uses SQL Server to create query optimization statistics for each index.

To view information on existing indexes, you can query the sys.indexes catalog view.

Limitations and Restrictions

You cannot create a rowstore index on a SQL Server PDW table when a columnstore index already exists. This behavior is different from SMP SQL Server which allows both rowstore and nonclustered columnstore indexes to co-exist on the same table.

Indexes cannot be created on views.

For information on minimum and maximum constraints on indexes, see Minimum and Maximum Values (SQL Server PDW).

Locking

Takes an exclusive lock on the TABLE object. Takes a shared lock on the DATABASE object.

Examples

A. Basic syntax

Copy image Copy Code
CREATE INDEX IX_VendorID
    ON ProductVendor (VendorID);
CREATE INDEX IX_VendorID
    ON dbo.ProductVendor (VendorID DESC, Name ASC, Address DESC);
CREATE INDEX IX_VendorID
    ON Purchasing..ProductVendor (VendorID);

B. Create a non-clustered index on a table in the current database

The following example creates a non-clustered index on the VendorID column of the ProductVendor table.

Copy image Copy Code
CREATE INDEX IX_ProductVendor_VendorID
    ON ProductVendor (VendorID);

C. Create a clustered index on a table in another database

The following example creates a non-clustered index on the VendorID column of the ProductVendor table in the Purchasing database.

Copy image Copy Code
CREATE CLUSTERED INDEX IX_ProductVendor_VendorID
    ON Purchasing..ProductVendor (VendorID);

D. Add a column to an index

The following example creates index IX_FF with two columns from the dbo.FactFinance table. The next statement demonstrates rebuilding that index with the same name and one more column.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE INDEX IX_FF ON dbo.FactFinance (
    FinanceKey ASC, DateKey ASC );

--Rebuild and add the OrganizationKey
CREATE INDEX IX_FF ON dbo.FactFinance (
    FinanceKey, DateKey, OrganizationKey DESC)
WITH ( DROP_EXISTING = ON );

See Also

Reference

ALTER INDEX (SQL Server PDW)
DROP INDEX (SQL Server PDW)


CREATE PROCEDURE (SQL Server PDW)

[image]

CREATE PROCEDURE (SQL Server PDW)
See Also Send Feedback

Creates a stored procedure for SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Create a stored procedure
CREATE { PROC | PROCEDURE } [ schema_name.] procedure_name
    [ { @parameter data_type } [ OUT | OUTPUT ] ] [ ,...n ]
AS { [ BEGIN ] sql_statement [;][ ,...n ] [ END ] }
[;]

Arguments

schema_name

The schema of the procedure.

procedure_name

The name of the procedure. Procedure names must comply with the rules for identifiers and must be unique within the schema.

Avoid the use of the sp_ prefix when naming procedures. This prefix is used by SQL Server to designate system procedures. Using the prefix can cause application code to break if there is a system procedure with the same name.

@parameter

A parameter declared in the procedure. Specify a parameter name by using the at sign (@) as the first character. The parameter name must comply with the rules for identifiers. Parameters are local to the procedure; the same parameter names can be used in other procedures.

OUT | OUTPUT

Indicates that the parameter is an output parameter. Use OUTPUT parameters to return values to the caller of the procedure. text, ntext, and image parameters cannot be used as OUTPUT parameters. An output parameter can be a cursor placeholder. A table-value data type cannot be specified as an OUTPUT parameter of a procedure.

{ [ BEGIN ] sql_statement [;] [ ...n ] [ END ] }

One or more SQL statements comprising the body of the procedure. You can use the optional BEGIN and END keywords to enclose the statements.

General Remarks

For a general description of stored procedures, see the SQL Server documentation at CREATE PROCEDURE (Transact-SQL).

Stored procedures can call stored procedures, and have up to 8 nested levels.

The following formats for calling stored procedures are permitted.

  • Calling a stored procedure with name determined at runtime: EXEC ('EXEC ' + @var);

  • Calling a stored procedure from within a stored procedure: CREATE sp_first AS EXEC sp_second; EXEC sp_third;

  • Executing a SQL string: EXEC (''SELECT * FROM sys.types'');

  • Executing a nested string: EXEC ('EXEC (''SELECT * FROM sys.types'')');

  • Calling a stored procedure by name without the EXEC or EXECUTE keyword if the statement is the first statement in the batch.

Limitations and Restrictions

When running CREATE PROCEDURE in a batch, CREATE PROCEDURE must be the first statement in the batch. This behavior is the same as SQL Server.

The following stored procedure features are not supported in SQL Server PDW.

  • Temporary stored procedures.

  • Numbered stored procedures.

  • Table valued parameters for stored procedures.

  • Extended stored procedures.

  • CLR stored procedures.

  • Read-only parameters for stored procedures.

  • Encrypted stored procedures.

  • INSERT … EXECUTE statements.

  • Execution of stored procedures under a different user context (the EXECUTE AS clause).

  • RAISERROR statements in stored procedures.

  • Default parameters.

  • The RETURN statement.

Permissions

Requires CREATE PROCEDURE permission in the database and ALTER permission on the schema in which the procedure is being created (dbo), or requires membership in the db_ddladmin fixed database role.

The sql_statement section of the stored procedure may require additional permissions.

Locking

Takes an exclusive lock on the procedure. Takes a shared lock on the DATABASE, SCHEMA, and SCHEMARESOLUTION object.

Examples

A. Create a Stored Procedure that runs a SELECT statement

This example shows the basic syntax for creating and running a procedure. When running a batch, CREATE PROCEDURE must be the first statement. For example, to create the following stored procedure in AdventureWorksPDW2012 , set the database context first, and then run the CREATE PROCEDURE statement.

Copy image Copy Code
--Run this statement first to set the database context.
USE
    AdventureWorksPDW2012
  ;

--Run CREATE PROCEDURE as the first statement in a batch.
CREATE PROCEDURE Get10TopResellers
AS
BEGIN
    SELECT TOP (10) r.ResellerName, r.AnnualSales
    FROM DimReseller AS r
    ORDER BY AnnualSales DESC, ResellerName ASC;
END
;

--Show 10 Top Resellers
EXEC Get10TopResellers;

--DROP the procedure if you need to re-run the example.
DROP PROCEDURE Get10TopResellers;

See Also

Reference

ALTER PROCEDURE (SQL Server PDW)
DROP PROCEDURE (SQL Server PDW)


CREATE SCHEMA (SQL Server PDW)

[image]

CREATE SCHEMA (SQL Server PDW)
See Also Send Feedback

Creates a schema in the current database. The CREATE SCHEMA transaction can also create tables and views within the new schema, and set GRANT, DENY, or REVOKE permissions on those objects.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE SCHEMA schema_name [ AUTHORIZATION owner_name ] [;]

Arguments

schema_name

Is the name by which the schema is identified within the database.

AUTHORIZATION owner_name

Specifies the name of the database-level principal that will own the schema. This principal may own other schemas, and may not use the current schema as its default schema.

Remarks

The database principal (user) that executes CREATE SCHEMA can specify another database principal as the owner of the schema being created. This requires additional permissions, as described in the "Permissions" section later in this topic.

The new schema is owned by one of the following database-level principals: database user, database role, or application role. Ownership of schema-contained objects can be transferred to any database-level principal, but the schema owner always retains CONTROL permission on objects within the schema.

Specifying the schema in Transact-SQL statements can slightly improve performance by removing some of the steps required to identify objects. Specifying the schema can also avoid errors from assuming the users default schema, and can make your code easier to understand and maintain.

Tip Tip

SQL Server PDW and SQL Server differ slightly in how a is resolved when the schema is omitted in a 3-part name referenced in a stored procedure. If schema is not provided, SQL Server will first try with the schema name which is the same as the name of the stored procedures schema (even if the reference is pointing to another database). If no such schema exists it will resolve to dbo. SQL Server PDW will first try with the schema name which is the same as the name of default schema for the user in the referenced database. This is the same result we would get in SQL Server if we executed statement as part of regular batch and not in stored procedure.

Implicit Schema and User Creation

In some cases a user can use a database without having a database user account (a database principal in the database). This can happen in the following situations:

  • A login has CONTROL SERVER privileges.

  • A Windows user does not have an individual database user account (a database principal in the database), but accesses a database as a member of a Windows group which has a database user account (a database principal for the Windows group).

When a user without a database user account creates an object without specifying an existing schema, a database principal and default schema will be automatically created in the database for that user. The created database principal and schema will have the same name as the name that user used when connecting to SQL Server (the SQL Server authentication login name or the Windows user name).

This behavior is necessary to allow users that are based on Windows groups to create and own objects. However it can result in the unintentional creation of schemas and users. To avoid implicitly creating users and schemas, whenever possible explicitly create database principals and assign a default schema. Or explicitly state an existing schema when creating objects in a database, using two or three-part object names.

Permissions

Requires CREATE SCHEMA permission on the database.

To specify another user as the owner of the schema being created, the caller must have IMPERSONATE permission on that user. If a database role is specified as the owner, the caller must have one of the following: membership in the role or ALTER permission on the role.

Locking

Takes an exclusive lock on the DATABASE, SCHEMA, and SCHEMARESOLUTION objects.

Examples

Example A

The following example creates schema Sales and then creates a table Sales.Region in that schema.

Copy image Copy Code
CREATE SCHEMA Sales;
GO;

CREATE TABLE Sales.Region
(Region_id int NOT NULL,
Region_Name char(5) NOT NULL)
WITH (DISTRIBUTION = REPLICATE);
GO

Example B

The following example creates a schema Production owned by Mary.

Copy image Copy Code
CREATE SCHEMA Production AUTHORIZATION [Contoso\Mary];
GO

See Also

Reference

ALTER SCHEMA (SQL Server PDW)
DROP SCHEMA (SQL Server PDW)
sys.schemas (SQL Server PDW)
Functions (SQL Server PDW)


CREATE STATISTICS (SQL Server PDW)

[image]

CREATE STATISTICS (SQL Server PDW)
See Also Send Feedback

Creates query optimization statistics on one or more columns of a SQL Server PDW table. The Control node uses statistics to improve the MPP distributed query plan for data movement operations, and the Compute nodes use statistics to improve the SMP query plan. Use this to improve query performance by pre-populating the Control node and the Compute nodes with user-defined single-column or multi-column statistics.

For more information, see Understanding Query Optimization Statistics (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE STATISTICS statistics_name
    ON [ database_name . [schema_name ] . | schema_name. ] table_name
    ( column_name  [ ,...n ] )
    [ WHERE <filter_predicate> ]
    [ WITH {
           FULLSCAN
           | SAMPLE number PERCENT
      }
    ]
[;]

<filter_predicate> ::=
    <conjunct> [AND <conjunct>]

<conjunct> ::=
    <disjunct> | <comparison>

<disjunct> ::=
        column_name IN (constant ,…)

<comparison> ::=
        column_name <comparison_op> constant

<comparison_op> ::=
    IS | IS NOT | = | <> | != | > | >= | !> | < | <= | !<

Arguments

statistics_name

The name of the statistics to create. See Object Naming Rules (SQL Server PDW) for information on supported statistics names.

[ database_name . [ schema_name ] . | schema_name . ] table_name

The one- to three-part name of the table that will contain the statistics.

column_name [ ,...n ]

One or more columns to be included in the statistics. The columns should be in priority order from left to right. Only the first column is used for creating the histogram. All columns are used for cross-column correlation statistics called densities.

WHERE <filter_predicate>

Specifies an expression for selecting a subset of rows to include when creating the statistics object. Statistics that are created with a filter predicate are called filtered statistics. The filter predicate uses simple comparison logic. Comparisons using NULL literals are not allowed with the comparison operators. Use the IS NULL and IS NOT NULL operators instead.

Here are some examples of filter predicates for the Production.BillOfMaterials table:

WHERE StartDate > '20000101' AND EndDate <= '20000630'

WHERE ComponentID IN (533, 324, 753)

WHERE StartDate IN ('20000404', '20000905') AND EndDate IS NOT NULL

FULLSCAN

Compute statistics by scanning all rows in the table. FULLSCAN and SAMPLE 100 PERCENT have the same results. FULLSCAN cannot be used with the SAMPLE option.

When neither FULLSCAN nor SAMPLE is specified, the query optimizer uses sampled data and computes the sample size by default.

SAMPLE number PERCENT

Specifies the approximate percentage of rows in the table for the query optimizer to use when it creates statistics. For PERCENT, number can be from 0 through 100. The actual percentage of rows the query optimizer samples might not match the percentage specified. For example, the query optimizer scans all rows on a data page.

SAMPLE cannot be used with the FULLSCAN option.

When neither SAMPLE nor FULLSCAN is specified, the query optimizer uses sampled data and computes the sample size by default. For regular tables, SQL Server determines the sample size. For external tables, SQL Server PDW computes the sample size by using a similar method.

SAMPLE is useful for special cases in which the query plan, based on default sampling, is not optimal. In most situations, it is not necessary to specify SAMPLE because the query optimizer already determines the sample size by default..

We recommend against specifying 0 PERCENT. When 0 PERCENT is specified, the statistics object is created but does not contain statistics data.

Permissions

Requires ALTER permission on the table, the table owner, or a member of the db_ddladmin fixed database role.

General Remarks

The query optimizer use statistics to estimate the cardinality of the query plan operations. This in turn enables the query optimizer to choose operations that will improve query performance.

This statement creates SMP SQL Server statistics on the Compute nodes and then merges the statistics. It stores the merged statistics on the Control node where the query optimizer uses them to make plan choices that make the data movement operations perform faster.

External table statistics are stored on the Control node. When creating external table statistics, SQL Server PDW imports the external table into a temporary SQL Server PDW table, and then creates the statistics. For sampled statistics, only the sampled rows are imported. If you have a large external table, it will be much faster to created sampled statistics.

Limitations and Restrictions

For information on minimum and maximum constraints on statistics, see Minimum and Maximum Values (SQL Server PDW).

Updating statistics is not supported on external tables. To update statistics on an external table, drop the statistics and then re-create the statistics.

SQL Server PDW does not automatically create and update statistics on the Control node. Running CREATE STATISTICS is the only way to populate statistics on the Control node.

The Compute nodes automatically create single-column statistics as needed for the SMP query plan. These are system-level statistics and are not exposed to user operations such as deleting statistics.

Locking

Takes an exclusive lock on the TABLE object. Takes a shared lock on the DATABASE object.

Performance

We recommend using CREATE STATISTICS to create single column statistics on columns that are used in join predicates, ORDER BY, WHERE, and GROUP BY clauses. This ensures the statistics are available to the Control node and to the Compute nodes when they choose the query plan.

To be in full control of creating, updating, and deleting statistics, consider creating single-column statistics right after creating the table. This avoids the possibility that a system generated statistic and also a user-defined statistic are generated on the same column; this would require double compute time to create the statistics and also to update statistics. We recommend creating single-column statistics on columns in join predicates, and also columns in WHERE, ORDER BY, and GROUP BY clauses.

Multi-column stats are not used for composite filters, which are filters on the same table. The cardinality estimator estimates the cardinality of composite filters with an approach called exponential backoff.

Multi-column statistics improve the query plan for:

  • Multiple joins on equality join predicates. Creating multi-column statistics for this situation can prevent SMP SQL Server from using nested loops in the query plan.

  • Multiple group by clauses. This improves the cardinality estimates for aggregations.

  • Distinct counts on multiple columns. This improves accuracy by reducing the cardinality estimate when the columns are correlated.

Examples

A. Create statistics on two columns

The following example creates the CustomerStats1 statistics, based on the CustomerKey and EmailAddress columns of the DimCustomer table. The statistics are created based on a statistically significant sampling of the rows in the Customer table.

Copy image Copy Code
CREATE STATISTICS CustomerStats1 ON DimCustomer (CustomerKey, EmailAddress);

B. Create statistics by using a full scan

The following example creates the CustomerStatsFullScan statistics, based on scanning all of the rows in the DimCustomer table.

Copy image Copy Code
CREATE STATISTICS CustomerStatsFullScan ON DimCustomer (CustomerKey, EmailAddress) WITH FULLSCAN;

C. Create statistics by specifying the sample percentage

The following example creates the CustomerStatsSampleScan statistics, based on scanning 50 percent of the rows in the DimCustomer table.

Copy image Copy Code
CREATE STATISTICS CustomerStatsSampleScan ON DimCustomer (CustomerKey, EmailAddress) WITH SAMPLE 50 PERCENT;

D. Create filtered statistics

The following example creates the filtered statistics ContactPromotion1. SQL Server PDW samples 50 percent of the data and then selects the rows with EmailPromotion equal to 2.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
IF EXISTS (SELECT name FROM sys.stats
    WHERE name = N'ContactPromotion1'
    AND object_id = OBJECT_ID(N'Person.Person'))
DROP STATISTICS Person.Person.ContactPromotion1;
GO
CREATE STATISTICS ContactPromotion1
    ON Person.Person (BusinessEntityID, LastName, EmailPromotion)
WHERE EmailPromotion = 2
WITH SAMPLE 50 PERCENT;
GO

See Also

Reference

UPDATE STATISTICS (SQL Server PDW)
DROP STATISTICS (SQL Server PDW)
STATS_DATE (PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Understanding Query Optimization Statistics (SQL Server PDW)


CREATE TABLE statements (SQL Server PDW)

[image]

CREATE TABLE statements (SQL Server PDW)
See Also Send Feedback

This section contains statement for creating tables in SQL Server PDW.

In This Section

  • CREATE TABLE (SQL Server PDW)

  • CREATE TABLE AS SELECT (SQL Server PDW)

  • CREATE EXTERNAL TABLE (SQL Server PDW)

  • CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW)

  • CREATE REMOTE TABLE AS SELECT (SQL Server PDW)

See Also

Concepts

Data Definition Statements (SQL Server PDW)
SQL Reference (SQL Server PDW)


CREATE EXTERNAL TABLE (SQL Server PDW)

[image]

CREATE EXTERNAL TABLE (SQL Server PDW)
See Also Send Feedback

This statement creates a SQL Server PDW external table that points to data in a Hadoop cluster or Azure blob storage.

Create an external table in order to:

  • Query Hadoop data from within SQL Server PDW.

  • Import and store Hadoop data into SQL Server PDW by using the CREATE TABLE AS SELECT statement.

For more information, see PolyBase for Hadoop and WASB Integration (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

--Create a new external table in SQL Server PDW
CREATE EXTERNAL TABLE [ database_name . [ schema_name ] . | schema_name. ] table_name
    ( <column_definition> [ ,...n ] )
    WITH (
        LOCATION = 'hdfs_folder_or_filepath',
        DATA_SOURCE = external_data_source_name,
        FILE_FORMAT = external_file_format_name
        [ , <reject_options> [ ,...n ] ]
    )
[;]

<reject_options> ::=
{
    | REJECT_TYPE = value | percentage
    | REJECT_VALUE = reject_value
    | REJECT_SAMPLE_VALUE = reject_sample_value

}

Arguments

database_name . [ schema_name ] . | schema_name. ] table_name

The one to three-part name of the table to create in SQL Server PDW. For an external table, only the table metadata is stored in SQL Server PDW. The metadata is stored in the Shell database on the Control node; no table data is stored on the Compute nodes.

<column_definition> [ ,...n ]

CREATE EXTERNAL TABLE allows one or more column definitions. Both CREATE EXTERNAL TABLE and CREATE TABLE use the same syntax for defining a column. An exception to this, you cannot use the DEFAULT CONSTRAINT on external tables.

For the full details about column definitions and their data types, see CREATE TABLE (SQL Server PDW).

For external tables that use the Hive RCFile format, the column and type definitions must map to the exact schema of the external file. When defining data type, use the following mappings between SQL and Hive data types. The types include all versions of Hive unless stated otherwise.

Note Note

When storing variations of the SQL Server date and time data types (date, time, smalldatetime, datetime, and datetime2) to a RCFile, the SQL Server data types are converted to the Unix timestamp data type (a 32-bit integer), limiting the date range to 1970-01-01 through 2038-1-19. To work around this limitation, cast the SQL Server type to a text type in the SELECT portion of CETAS. The resulting EXTERNAL TABLE definition will have the text type specified for that column. Cast the type back into a SQL Server datetime type when selecting from it.

SQL Data Type

Hive Data Type

Hadoop/Java Data Type

Comments

tinyint

tinyint

ByteWritable

For unsigned numbers only.

smalling

smallint

ShortWritable

int

int

IntWritable

bigint

bigint

LongWritable

bit

boolean

BooleanWritable

float

double

DoubleWritable

real

float

FloatWritable

money

double

DoubleWritable

smallmoney

double

DoubleWritable

nchar

string

text

nvarchar

string

Text

char

string

Text

varchar

string

Text

binary

binary

BytesWritable

Applies to Hive 0.8 and later.

varbinary

binary

BytesWritable

Applies to Hive 0.8 and later.

date

timestamp

TimestampWritable

smalldatetime

timestamp

TimestampWritable

datetime2

timestamp

TimestampWritable

datetime

timestamp

TimestampWritable

time

timestamp

TimestampWritable

decimal

decimal

BigDecimalWritable

Applies to Hive0.11 and later.

LOCATION = 'hdfs_folder_or_filepath'

Specifies the folder or the file path and file name where the data file is located in the Hadoop Cluster. The location starts from the root folder of the Hadoop Cluster; the root folder is specified in CREATE EXTERNAL DATA SOURCE (SQL Server PDW).

If you specify LOCATION to be a folder, a PolyBase query that selects from the external table will return files from the folder and all of the subfolders. Just like Hadoop, PolyBase does not return hidden folders. It also does not return files for which the file name begins with an underline (_) or a period (.).

In this example, if LOCATION='/webdata/', a query will return rows from mydata.txt and mydata2.txt. It will not return mydata3.txt because it is a subfolder of a hidden folder. It will not return _hidden.txt because it is a hidden file.

Recursive data for external tables

DATA_SOURCE = external_data_source_name

Specifies the name of the external data source that contains the location where the external data is stored or will be stored. The location is either a Hadoop Cluster or Windows Azure blob storage. To create an external data source, use CREATE EXTERNAL DATA SOURCE (SQL Server PDW).

FILE_FORMAT = external_file_format_name

Specifies the name of the external file format that contains the format for the Hadoop data file. To create an external file format, use CREATE EXTERNAL FILE FORMAT (SQL Server PDW).

Reject Options

The reject options do not apply when CREATE EXTERNAL TABLE runs. Instead, they are stored with the table metadata so that SQL Server PDW can use them at a later time when it imports data from the external table into SQL Server PDW. For example, when a future SELECT statement or CREATE TABLE AS SELECT statement selects data from the external table, SQL Server PDW will use the reject options to determine the number or percentage of rows that can fail to import before it stops the import.

REJECT_VALUE = reject_value

Specifies the value or the percentage of rows that can fail to import before SQL Server PDW halts the import.

REJECT_TYPE = value | percentage

Clarifies whether the REJECT_VALUE option is specified as a literal value or a percentage.

value

REJECT_VALUE is a literal value, not a percentage. SQL Server PDW will stop importing rows from the external data file when the number of failed rows exceeds reject_value.

For example, if REJECT_VALUE = 5 and REJECT_TYPE = value, SQL Server PDW will stop importing rows after 5 rows have failed to import.

percentage

REJECT_VALUE is a percentage, not a literal value. SQL Server PDW will stop importing rows from the external data file when the percentage of failed rows exceeds reject_value. The percentage of failed rows is calculated at intervals.

REJECT_SAMPLE_VALUE = reject_sample_value

Required when REJECT_TYPE = percentage. It specifies the number of rows to attempt to import before SQL Server PDW recalculates the percentage of failed rows.

For example, if REJECT_SAMPLE_VALUE = 1000, SQL Server PDW will calculate the percentage of failed rows after it has attempted to import 1000 rows from the external data file. If the percentage of failed rows is less than reject_value, SQL Server PDW will attempt to load another 1000 rows. SQL Server PDW continues to recalculate the percentage of failed rows after it attempts to import each additional 1000 rows.

Note Note

Since SQL Server PDW computes the percentage of failed rows at intervals, the actual percentage of failed rows can exceed reject_value.

Example:

This example shows how the three REJECT options interact with each other. For example, if REJECT_TYPE = percentage, REJECT_VALUE = 30, and REJECT_SAMPLE_VALUE = 100, the following scenario could occur:

  • SQL Server PDW attempts to load the first 100 rows; 25 fail and 75 succeed.

  • Percent of failed rows is calculated as 25%, which is less than the reject value of 30%. So, no need to halt the load.

  • SQL Server PDW attempts to load the next 100 rows; this time 25 succeed and 75 fail.

  • Percent of failed rows is recalculated as 50%. The percentage of failed rows has exceeded the 30% reject value.

  • The load fails with 50% failed rows after attempting to load 200 rows, which is larger than the specified 30% limit..

Permissions

The CREATE EXTERNAL TABLE AS SELECT statement requires the following permissions:

  • The database user must have membership in the db_ddladmin fixed database role or the ALTER SCHEMA permission on the local schema that will contain the new table.

  • The database user must have membership in the db_ddladmin fixed database role or the CREATE TABLE permission.

  • The database user must have SELECT permission on any objects referenced in the select_criteria.

  • The login must have the ADMINISTER BULK OPERATIONS permission.

  • The login must have the ALTER ANY EXTERNAL DATA SOURCE permission.

  • The login must have the ALTER ANY EXTERNAL FILE FORMAT permission.

  • The login must have write permission on the Hadoop Cluster folder.

Error Handling

When CREATE EXTERNAL TABLE runs, SQL Server PDW attempts to connect to the external Hadoop cluster. If the connection fails, the command will fail and the external table will not be created. It can take a minute or more for the command to fail since SQL Server PDW retries the connection 3 times.

General Remarks

The row data for an external table is stored in the Hadoop Distributed File System (HDFS) with text-delimited or RcFile format. These data files are created and managed by your own processes.

The external Hadoop data is imported into SQL Server PDW when the data is required by a query predicate, or when a CREATE TABLE AS SELECT statement selects from the external table. During query execution, the data is stored in a SQL Server PDW temporary table. After the query finishes, SQL Server PDW removes and deletes the temporary table. During CREATE TABLE AS SELECT, SQL Server PDW imports and stores the data into a permanent tablein SQL Server PDW.

When executing queries on external Hadoop tables, SQL Server PDW performs predicate pushdown to compute parts of the query in Hadoop before copying data into SQL Server PDW. To perform predicate pushdown, SQL Server PDW issues map-reduce jobs to run on Hadoop data by using the job tracker ID specified in the external data source. All SQL Server PDW supported data types can be part of a predicate or expression to be evaluated on Hadoop.

For more information, see Queries on External Tables With Predicate Pushdown (SQL Server PDW).

In AU2, by default a PolyBase query on an external table folder returned data stored in the root folder only. Beginning with AU3, by default a query on an external table returns data from the root folder and all of the subfolders. The root folder is specified by the LOCATION parameter in CREATE EXTERNAL TABLE. To avoid unwanted results, verify that subfolders in external file folders do not contain extra data that should not be returned in query results. For backwards compatibility, your appliance administrator can change the default setting to read data from only the root folder of the external file location. To do this, set <polybase.recursive.traversal> to 'false' in the appliance configuration file called core-site.xml.

Limitations and Restrictions

By design, you cannot export data to an external table that was created with the CREATE EXTERNAL TABLE statement. To export data to a new external table, use CREATE EXTERNAL TABLE AS SELECT.

You can have multiple external tables among the SQL Server PDW user databases and they can each be connected to the same or different Hadoop cluster. When connections exist to different clusters, all Hadoop clusters must be running the same version of Hadoop.

Since the data for an external table resides off the appliance, it is not under the control of SQL Server PDW, and can be changed or removed at any time by an external process. As a result:

  • CREATE TABLE, DROP TABLE, CREATE STATISTICS, DROP STATISTICS, CREATE VIEW, and DROP VIEW are the only DDL operations allowed on external Tables.

  • External table columns cannot use the DEFAULT constraint.

  • A query will fail if the connection to the Hadoop cluster fails.

  • Query results against an external table are not guaranteed to be deterministic. The same query can return different results each time it runs against an external table.

  • A query can fail if the off-appliance table data is removed or relocated.

PolyBase can consume a maximum of 33k files per folder when running 32 concurrent PolyBase queries. This maximum number includes both files and subfolders in each HDFS folder. If the degree of concurrency is less than 32, a user can run PolyBase queries against folders in HDFS which contain more than 33k files. Microsoft recommends users of Hadoop and PolyBase keep file paths short and use no more than 30k files per HDFS folder. When too many files are referenced a JVM out-of-memory exception occurs.

Locking

Takes a shared lock on the SCHEMARESOLUTION object.

Examples

A. Create an external table with data in text-delimited format.

This example shows all the steps required to create an external table that has data formatted in text-delimited files. It defines an external data source mydatasource and an external file format myfileformat. These sever-level objects are then referenced in the CREATE EXTERNAL TABLE statement. For more information, see CREATE EXTERNAL DATA SOURCE (SQL Server PDW) and CREATE EXTERNAL FILE FORMAT (SQL Server PDW).

Copy image Copy Code
CREATE EXTERNAL DATA SOURCE mydatasource
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://xxx.xxx.xxx.xxx:8020'
)

CREATE EXTERNAL FILE FORMAT myfileformat
WITH (
    FORMAT_TYPE = DELIMITEDTEXT,
    FORMAT_OPTIONS (FIELD_TERMINATOR ='|')
);

CREATE EXTERNAL TABLE ClickStream (
    url varchar(50),
    event_date date,
    user_IP varchar(50)
)
WITH (
        LOCATION='/webdata/employee.tbl',
        DATA_SOURCE = mydatasource,
        FILE_FORMAT = myfileformat
    )
;

B. Create an external table with data in RCFile format.

This example shows all the steps required to create an external table that has data formatted as RCFiles. It defines an external data source mydatasource_rc and an external file format myfileformat_rc. These sever-level objects are then referenced in the CREATE EXTERNAL TABLE statement. For more information, see CREATE EXTERNAL DATA SOURCE (SQL Server PDW) and CREATE EXTERNAL FILE FORMAT (SQL Server PDW).

Note Note

SQL Server PDW requires that all data sources are of the same format type. You cannot have some data sources with text-delimited file formats and some with RCFILE formats.

Copy image Copy Code
CREATE EXTERNAL DATA SOURCE mydatasource_rc
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://xxx.xxx.xxx.xxx:8020'
)

CREATE EXTERNAL FILE FORMAT myfileformat_rc
WITH (
    FORMAT = RCFILE,
    SERDE_METHOD = 'org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe'
)
;

CREATE EXTERNAL TABLE ClickStream_rc (
    url varchar(50),
    event_date date,
    user_ip varchar(50)
)
WITH (
        LOCATION='/webdata/employee_rc.tbl',
        DATA_SOURCE = mydatasource_rc,
        FILE_FORMAT = myfileformat_rc
    )
;

C. Create an external table with data in ORC format.

This example shows all the steps required to create an external table that has data formatted as ORC files. It defines an external data source mydatasource_orc and an external file format myfileformat_orc. These sever-level objects are then referenced in the CREATE EXTERNAL TABLE statement. For more information, see CREATE EXTERNAL DATA SOURCE (SQL Server PDW) and CREATE EXTERNAL FILE FORMAT (SQL Server PDW).

Note Note

SQL Server PDW requires that all data sources are of the same format type. You cannot have some data sources with text-delimited file formats, some with RCFILE formats, and some with ORC file formats.

Copy image Copy Code
CREATE EXTERNAL DATA SOURCE mydatasource_orc
WITH (
    TYPE = HADOOP,
    LOCATION = 'hdfs://xxx.xxx.xxx.xxx:8020'
)

CREATE EXTERNAL FILE FORMAT myfileformat_orc
WITH (
    FORMAT = ORC,
    COMPRESSION = 'org.apache.hadoop.io.compress.SnappyCodec'
)
;

CREATE EXTERNAL TABLE ClickStream_orc (
    url varchar(50),
    event_date date,
    user_ip varchar(50)
)
WITH (
        LOCATION='/webdata/',
        DATA_SOURCE = mydatasource_orc,
        FILE_FORMAT = myfileformat_orc
    )
;

D. Retrieving HDFS data in SQL Server PDW

Clickstream is an external table that connects to the employee.tbl delimited text file on a Hadoop cluster. The following SQL Server PDW query runs against data on a Hadoop cluster.

Copy image Copy Code
SELECT TOP 10 (url) FROM ClickStream WHERE user_ip = 'xxx.xxx.xxx.xxx'
;

E. Join external tables

Copy image Copy Code
SELECT url.description
FROM ClickStream cs
JOIN UrlDescription url ON cs.url = url.name
WHERE cs.url = 'msdn.microsoft.com'
;

F. Join HDFS data with PDW data

Copy image Copy Code
SELECT cs.user_ip FROM ClickStream cs
JOIN User u ON cs.user_ip = u.user_ip
WHERE cs.url = 'www.microsoft.com'
;

G. Import row data from HDFS into a distributed PDW Table

Copy image Copy Code
CREATE TABLE ClickStream_PDW
WITH ( DISTRIBUTION = HASH (url) )
AS SELECT url, event_date, user_ip FROM ClickStream
;

H. Import row data from HDFS into a replicated PDW Table

Copy image Copy Code
CREATE TABLE ClickStream_PDW
WITH ( DISTRIBUTION = REPLICATE )
AS SELECT url, event_date, user_ip
FROM ClickStream
;

See Also


CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW)

[image]

CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW)
See Also Send Feedback

Creates a SQL Server PDW external table and then exports, in parallel, the results of a Transact-SQL SELECT statement to Hadoop.

Use the CREATE EXTERNAL TABLE AS SELECT (CETAS) statement to:

  • Export a SQL Server PDW table to Hadoop.

  • Import multiple Hadoop tables into SQL Server PDW, join them, and then export the results in parallel back to Hadoop.

  • Join a Hadoop table and a SQL Server PDW table, and then export the results in parallel back to Hadoop.

For more information, see PolyBase for Hadoop and WASB Integration (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE EXTERNAL TABLE [ database_name . [ schema_name ] . | schema_name. ] table_name
    WITH (
        LOCATION = 'hdfs_folder_or_filepath',
        DATA_SOURCE = external_data_source_name,
        FILE_FORMAT = external_file_format_name
        [ , <reject_options> [ ,...n ] ]
    )
    AS <select_statement>
[;]

<reject_options> ::=
{
    | REJECT_TYPE = value | percentage
    | REJECT_VALUE = reject_value
    | REJECT_SAMPLE_VALUE = reject_sample_value
}

<select_statement> ::=
    [ WITH <common_table_expression> [ ,...n ] ]
    SELECT <select_criteria>

Arguments

[ database_name . [ schema_name ] . | schema_name. ] table_name

The one to three-part name of the table to create in SQL Server PDW. For an external table, only the table metadata is stored in SQL Server PDW. The metadata is stored in the Shell database on the Control node; no table data is stored on the Compute nodes.

LOCATION = 'hdfs_folder_or_filepath'

Specifies the folder or the filepath where the data file is located in the Hadoop Cluster. The location starts from the root folder of the Hadoop Cluster. The Hadoop Cluster is specified in the external data source, created by using CREATE EXTERNAL DATA SOURCE (SQL Server PDW).

If you specify a folder with no file name, SQL Server PDW imports data from all files in the folder.

Examples:

  • '/webdata/'

  • '/webdata/abc/'

  • '/webdata/abc/mydata.txt'

DATA_SOURCE = external_data_source_name

Specifies the name of the external data source that contains the location where the external data is stored or will be stored. The location is either a Hadoop Cluster or Azure blob storage. To create an external data source, use CREATE EXTERNAL DATA SOURCE (SQL Server PDW).

FILE_FORMAT = external_file_format_name

Specifies the name of the external file format that contains the format for the Hadoop data file. To create an external file format, use CREATE EXTERNAL FILE FORMAT (SQL Server PDW).

Reject Options

The reject options do not apply when this statement is run. They need to be specified with CREATE EXTERNAL TABLE AS SELECT so that SQL Server PDW can use them at a later time when it imports data from the external table into SQL Server PDW. For example, when a future SELECT statement or CREATE TABLE AS SELECT statement selects data from the external table, SQL Server PDW will use the reject options to determine the number or percentage of rows that can fail to import before it stops the import.

REJECT_VALUE = reject_value

Specifies the value or the percentage of rows that can fail to import before SQL Server PDW halts the import.

REJECT_TYPE = value | percentage

Clarifies whether the REJECT_VALUE option is specified as a literal value or a percentage.

value

REJECT_VALUE is a literal value, not a percentage. SQL Server PDW will stop importing rows from the external data file when the number of failed rows exceeds reject_value.

For example, if REJECT_VALUE = 5 and REJECT_TYPE = value, SQL Server PDW will stop importing rows after 5 rows have failed to import.

percentage

REJECT_VALUE is a percentage, not a literal value. SQL Server PDW will stop importing rows from the external data file when the percentage of failed rows exceeds reject_value. The percentage of failed rows is calculated at intervals.

REJECT_SAMPLE_VALUE = reject_sample_value

Required when REJECT_TYPE = percentage. It specifies the number of rows to attempt to import before SQL Server PDW recalculates the percentage of failed rows.

For example, if REJECT_SAMPLE_VALUE = 1000, SQL Server PDW will calculate the percentage of failed rows after it has attempted to import 1000 rows from the external data file. If the percentage of failed rows is less than reject_value, SQL Server PDW will attempt to load another 100 rows. SQL Server PDW continues to recalculate the percentage of failed rows after it attempts to import each additional 1000 rows.

Note Note

Since SQL Server PDW computes the percentage of failed rows at intervals, the actual percentage of failed rows can exceed reject_value.

Example:

This example shows how the three REJECT options interact with each other. For example, if REJECT_TYPE = percentage, REJECT_VALUE = 30, and REJECT_SAMPLE_VALUE = 100, the following scenario could occur:

  • SQL Server PDW attempts to load the first 100 rows; 25 fail and 75 succeed.

  • Percent of failed rows is calculated as 25%, which is less than the reject value of 30%. So, no need to halt the load.

  • SQL Server PDW attempts to load the next 100 rows; this time 25 succeed and 75 fail.

  • Percent of failed rows is recalculated as 50%. The percentage of failed rows has exceeded the 30% reject value.

  • The load fails with 50% failed rows after attempting to load 200 rows, which is larger than the specified 30% limit.

WITH common_table_expression

Specifies a temporary named result set, known as a common table expression (CTE). For more information, see WITH common_table_expression (SQL Server PDW).

SELECT <select_criteria>

Populates the new table with the results from a SELECT statement. select_criteria is the body of the SELECT statement that determines which data to copy to the new table. For information about SELECT statements, see SELECT (SQL Server PDW).

Permissions

The CREATE EXTERNAL TABLE AS SELECT statement requires the following permissions:

  • The database user must have membership in the db_ddladmin fixed database role or the ALTER SCHEMA permission on the local schema that will contain the new table.

  • The database user must have membership in the db_ddladmin fixed database role or the CREATE TABLE permission.

  • The database user must have SELECT permission on any objects referenced in the select_criteria.

  • The login must have the ADMINISTER BULK OPERATIONS permission.

  • The login must have the ALTER ANY EXTERNAL DATA SOURCE permission.

  • The login must have the ALTER ANY EXTERNAL FILE FORMAT permission.

  • The login must have write permission on the Hadoop Cluster folder.

Error Handling

When CREATE EXTERNAL TABLE AS SELECT exports data to a Hadoop text-delimited file, there is no rejection file for rows that fail to export.

When creating the external table, SQL Server PDW attempts to connect to the external Hadoop cluster. If the connection fails, the command will fail and the external table will not be created. It can take a minute or more for the command to fail since SQL Server PDW retries the connection 3 times.

If CREATE EXTERNAL TABLE AS SELECT is cancelled or fails, SQL Server PDW will make a one-time attempt to remove any new files and directories that were created on the external data source.

SQL Server PDW will report any Java errors that occur on the Hadoop Cluster during the data export.

General Remarks

As a prerequisite for creating an external table, the PDW region administrator needs to configure hadoop connectivity. For more information, see Configure PolyBase Connectivity to External Data (Analytics Platform System) in the Appliance Operations Guide.

After the CETAS statement finishes, you can run SQL queries on the external table. These operations will import data into SQL Server PDW for the duration of the query unless you import by using the CREATE TABLE AS SELECT statement.

The external table name and definition are stored in the SQL Server PDW metadata. The results of the SELECT statement are written to the file on an external Hadoop Cluster.

The Hadoop text files are named table_name. ID.txt where ID is an incremental identifier. For example, if the external table name is JanuaryClicks, the SQL Server PDW table name is JanuaryClicks and the Hadoop file name is JanuaryClicks.1.txt. If you delete the SQL Server PDW JanuaryClicks table, the Hadoop JanuaryClicks.1.txt remains. If you run CETAS again to create January Clicks, this time the Hadoop file is called JanuaryClicks.2.txt.

The CETAS statement always creates a non-partitioned table, even if the source table is partitioned.

For query plans, created with EXPLAIN, SQL Server PDW has the following operations that are used specifically for external tables:

  • External shuffle move

  • External broadcast move

  • External partition move

Limitations and Restrictions

Since external table data resides off the appliance, backup and restore will only backup and restore the table definition stored in the Shell database on the Control node.

SQL Server PDW does not verify the connection to the Hadoop Cluster when restoring a database backup that contains an external table. If the original source HDFS file is not accessible, the metadata restore of the external table will still succeed, but SELECT operations that import data from the external table will fail.

Since the external table data is stored off of the appliance, SQL Server PDW does not guarantee any data consistency between SQL Server PDW and the external HDFS data source file. It is the customer’s responsibility to maintain the consistency between HDFS data and SQL Server PDW data.

DML operations are not supported on external tables. For example, you cannot use SQL Server PDW to insert, delete, or modify the HDFS data.

CREATE TABLE, DROP TABLE, CREATE STATISTICS, DROP STATISTICS, CREATE VIEW, and DROP VIEW are the only DDL operations allowed on external Tables.

PolyBase can consume a maximum of 33k files per folder when running 32 concurrent PolyBase queries. This maximum number includes both files and subfolders in each HDFS folder. If the degree of concurrency is less than 32, a user can run PolyBase queries against folders in HDFS which contain more than 33k files. Microsoft recommends users of Hadoop and PolyBase keep file paths short and use no more than 30k files per HDFS folder. When too many files are referenced a JVM out-of-memory exception occurs.

SET ROWCOUNT (SQL Server PDW) has no effect on this statement. To achieve a similar behavior, use TOP (SQL Server PDW).

When CREATE EXTERNAL TABLE AS SELECT selects from an RCFile, the column values in the RCFile must not contain the pipe '|' character.

Locking

Takes a shared lock on the SCHEMARESOLUTION object.

Examples

A. Create a table using CREATE EXTERNAL TABLE AS SELECT (CETAS)

The following example creates a new external table named hdfsCustomer, using the column definitions and data from the source table dimCustomer.

The table definition is stored in SQL Server PDW, and the results of the SELECT statement are exported to the '/pdwdata/customer.tbl' file on the Hadoop external data source customer_ds. The file is formatted according to the external file format customer_ff.

The file name is generated by SQL Server PDW, and contains the query ID for ease of aligning the file with the query that generated it.

The path hdfs://xxx.xxx.xxx.xxx:5000/files/ preceding the Customer directory must already exist. However, if the Customer directory does not exist, SQL Server PDW will create the directory.

Note Note

This example specifies for 5000. If the port is not specified, SQL Server PDW uses 8020 as the default port.

The resulting Hadoop location and file name will be hdfs:// xxx.xxx.xxx.xxx:5000/files/Customer/ QueryID_YearMonthDay_HourMinutesSeconds_FileIndex.txt..

Copy image Copy Code
USE AdventureWorksPDW2012;
CREATE EXTERNAL TABLE hdfsCustomer
WITH (
        LOCATION='/pdwdata/customer.tbl',
        DATA_SOURCE = customer_ds,
        FILE_FORMAT = customer_ff
    )
) AS SELECT * FROM dimCustomer;

B. Use a Query Hint with CREATE EXTERNAL TABLE AS SELECT (CETAS)

This query shows the basic syntax for using a query join hint with the CETAS statement. After the query is submitted to the Control node, SQL Server, running on the Compute nodes, will apply the hash join strategy when generating the SQL Server query plan. For more information on join hints and how to use the OPTION clause, see OPTION (SQL Server PDW).

Note Note

This example specifies for 5000. If the port is not specified, SQL Server PDW uses 8020 as the default port.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE EXTERNAL TABLE dbo.FactInternetSalesNew
WITH
    (
        LOCATION = '/files/Customer',
        DATA_SOURCE = customer_ds,
        FILE_FORMAT = customer_ff
    )
AS SELECT T1.* FROM dbo.FactInternetSales T1 JOIN dbo.DimCustomer T2
ON ( T1.CustomerKey = T2.CustomerKey )
OPTION ( HASH JOIN );

See Also

Reference

CREATE EXTERNAL TABLE (SQL Server PDW)
CREATE TABLE (SQL Server PDW)
DROP TABLE (SQL Server PDW)
ALTER TABLE (SQL Server PDW)


CREATE REMOTE TABLE AS SELECT (SQL Server PDW)

[image]

CREATE REMOTE TABLE AS SELECT (SQL Server PDW)
See Also Send Feedback

Selects data from a SQL Server PDW database and copies that data to a new table in a SMP SQL Server database on a remote server. SQL Server PDW uses the appliance, with all the benefits of MPP query processing, to select the data for the remote copy. Use this for scenarios that require SQL Server functionality.

To configure the remote server, see Remote Table Copy (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE REMOTE TABLE [ database_name . [ schema_name ] . | schema_name. ] table_name
    AT ('<connection_string>')
    [ WITH ( BATCH_SIZE = batch_size ) ]
    AS <select_statement>
[;]

<connection_string> ::=
    Data Source = { IP_address | hostname } [, port ]; User ID = user_name ;Password = password;

<select_statement> ::=
    [ WITH <common_table_expression> [ ,...n ] ]
    SELECT <select_criteria>

Arguments

database_name

The database to create the remote table in. database_name is a SQL Server database. Default is the default database for the user login on the destination SQL Server instance.

schema_name

The schema for the new table. Default is the default schema for the user login on the destination SQL Server instance.

table_name

The name of the new table. For details on permitted table names, see Object Naming Rules (SQL Server PDW).

The remote table is created as a heap. It does not have check constraints or triggers. The collation of the remote table columns is the same as the collation of the source table columns. This applies to columns of type char, nchar, varchar, and nvarchar.

connection_string

A character string that specifies the Data Source, User ID, and Password parameters for connecting to the remote server and database.

The connection string is a semicolon-delimited list of key and value pairs. Keywords are not case-sensitive. Spaces between key and value pairs are ignored. However, values may be case-sensitive, depending on the data source.

Data Source

The parameter that specifies the name or IP address, and TCP port number for the remote SMP SQL Server.

hostname or IP_address

Name of the remote server computer or the IPv4 address of the remote server. IPv6 addresses are not supported. You can specify a SQL Server named instance in the format Computer_Name\Instance_Name or IP_address\Instance_Name. The server must be remote and therefore cannot be specified as (local).

TCP port number

The TCP port number for the connection. You can specify a TCP port number from 0 to 65535 for an instance of SQL Server that is not listening on the default port 1433. For example: ServerA,1450 or 10.192.14.27,1435

Note Note

We recommend connecting to a remote server by using the IP address. Depending on your network configuration, connecting by using the computer name might require additional steps to use your non-appliance DNS server to resolve the name to the correct server. This step is not necessary when connecting with an IP address. For more information, see Use a DNS Forwarder to Resolve Non-Appliance DNS Names (Analytics Platform System).

user_name

A valid SQL Server authentication login name. Maximum number of characters is 128.

password

The login password. Maximum number of characters is 128.

batch_size

The maximum number of rows per batch. SQL Server PDW sends rows in batches to the destination server. Batch_size is a positive integer >= 0. Default is 0.

WITH common_table_expression

Specifies a temporary named result set, known as a common table expression (CTE). For more information, see WITH common_table_expression (SQL Server PDW).

SELECT <select_criteria>

The query predicate that specifies which data will populate the new remote table. For information on the SELECT statement, see SELECT (SQL Server PDW).

Permissions

Requires:

  • SELECT permission on each object in the SELECT clause.

  • Requires CREATE TABLE permission on the destination SMP database.

  • Requires ALTER, INSERT, and SELECT permissions on the destination SMP schema.

Error Handling

If copying data to the remote database fails, SQL Server PDW will abort the operation, log an error, and attempt to delete the remote table. SQL Server PDW does not guarantee a successful cleanup of the new table.

Limitations and Restrictions

Remote Destination Server:

  • TCP is the default and only supported protocol for connecting to a remote server.

  • The destination server must be a non-appliance server. CREATE REMOTE TABLE cannot be used to copy data from one appliance to another.

  • The CREATE REMOTE TABLE statement only creates new tables. Therefore, the new table cannot already exist. The remote database and schema must already exist.

  • The remote server must have space available to store the data that is transferred from the appliance to the SQL Server remote database.

SELECT statement:

  • The ORDER BY and TOP clauses are not supported in the select criteria.

  • CREATE REMOTE TABLE cannot be run inside an active transaction or when the AUTOCOMMIT OFF setting is active for the session.

SET ROWCOUNT (SQL Server PDW) has no effect on this statement. To achieve a similar behavior, use TOP (SQL Server PDW).

Locking Behavior

After creating the remote table, the destination table is not locked until the copy starts. Therefore, it is possible for another process to delete the remote table after it is created and before the copy starts. When this occurs, SQL Server PDW will generate an error and the copy will fail.

Metadata

Use sys.dm_pdw_dms_workers (SQL Server PDW) to view the progress of copying the selected data to the remote SMP server. Rows with type PARALLEL_COPY_READER contain this information.

Security

CREATE REMOTE TABLE uses SQL Server Authentication to connect to the remote SQL Server instance; it does not use Windows Authentication.

The SQL Server PDW external facing network must be firewalled, with exception of SQL Server ports, administrative ports, and management ports.

To help prevent accidental data loss or corruption, the user account that is used to copy from the appliance to the destination database should have only the minimum required permissions on the destination database.

Connection settings allow you to connect to the SMP SQL Server instance with SSL protecting user name and password data, but with actual data being sent unencrypted in clear text. When this occurs, a malicious user could intercept the CREATE REMOTE TABLE statement text, which contains the SQL Server user name and password to log onto the SMP SQL Server instance. To avoid this risk, use data encryption on the connection to the SMP SQL Server instance.

Examples

A. Creating a remote table

This example creates a SQL Server SMP remote table called MyOrdersTable on database OrderReporting and schema Orders. The OrderReporting database is on a server named SQLA that listens on the default port 1433. The connection to the server uses the credentials of the user David, whose password is e4n8@3.

Copy image Copy Code
CREATE REMOTE TABLE OrderReporting.Orders.MyOrdersTable
AT ( 'Data Source = SQLA, 1433; User ID = David; Password = e4n8@3;' )
AS SELECT <select_criteria>;

B. Querying the sys.dm_pdw_dms_workers DMV for remote table copy status

This query shows how to view copy status for a remote table copy.

Copy image Copy Code
SELECT * FROM sys.dm_pdw_dms_workers
WHERE type = 'PARALLEL_COPY_READER';

C. Using a query join hint with CREATE REMOTE TABLE

This query shows the basic syntax for using a query join hint with CREATE REMOTE TABLE. After the query is submitted to the Control node, SQL Server, running on the Compute nodes, will apply the hash join strategy when generating the SQL Server query plan. For more information on join hints and how to use the OPTION clause, see OPTION (SQL Server PDW).

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE REMOTE TABLE OrderReporting.Orders.MyOrdersTable
AT ( 'Data Source = SQLA, 1433; User ID = David; Password = e4n8@3;' )
    AS SELECT T1.* FROM OrderReporting.Orders.MyOrdersTable T1
    JOIN OrderReporting.Order.Customer T2
    ON T1.CustomerID=T2.CustomerID OPTION (HASH JOIN);

See Also


CREATE TABLE (SQL Server PDW)

[image]

CREATE TABLE (SQL Server PDW)
See Also Send Feedback

Creates a new table in SQL Server PDW. Use the CLUSTERED COLUMNSTORE INDEX table option to store the table as an xVelocity memory-optimized clustered (and updateable) columnstore index. Use additional table options to specify the distribution method and the partitioning scheme.

Note Note

We view the clustered columnstore index as the standard for storing tables in SQL Server PDW, and expect it will be used in most scenarios.

For more information, see Clustered Columnstore Indexes (SQL Server PDW).

For copying data, use the CREATE TABLE AS SELECT (CTAS) statement to create and populate a table with the results from a SELECT statement. For example, you can use the CTAS statement to redistribute a table using a different distribution column. For more information, see CREATE TABLE AS SELECT (SQL Server PDW).

For storing data, replicate the table to each Compute node, or hash-distribute the table across all of the Compute nodes. To distribute a table, you must specify a distribution column; SQL Server PDW assigns each row to be stored in one distribution of one Compute node by hashing the distribution column value. For more information about distributed and replicated tables, see Distributed and Replicated Tables (SQL Server PDW).

For managing data, create table partitions that are logical subsets of rows tracked with metadata. To partition a table, you specify one partitioning column and a set of boundary values. This does not affect which distribution or Compute node where the data are stored. Instead, it allows you to perform data management operations on logical subsets of the table. For example, if you have monthly data, you can partition the table by month, and then archive an older month by switching its partitions to an archive table.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

Create a new table.
CREATE TABLE [ database_name . [ schema_name ] . | schema_name. ] table_name
    (
        { column_name <data_type>
        [ COLLATE Windows_collation_name ]
        [ NULL | NOT NULL ]
        [ [ CONSTRAINT constraint_name ] DEFAULT constant_expression  ] }
        [ ,...n ]
    )
    [ WITH ( <table_option> [ ,...n ] ) ]
[;]

Create a new temporary table.
CREATE TABLE [ database_name . [ dbo ] . | dbo. ] #table_name
    (
        { column_name <data_type>
        [ COLLATE Windows_collation_name ]
        [ NULL | NOT NULL ] }
        [ ,...n ]
    )
    WITH ( LOCATION = USER_DB [, <table_option> [ ,...n ] ] )
[;]

<table_option> ::=
    CLUSTERED COLUMNSTORE INDEX
    | CLUSTERED INDEX ( { index_column_name [ ASC | DESC ] } [ ,...n ] )
    | DISTRIBUTION = { HASH ( distribution_column_name ) | REPLICATE }
    | PARTITION ( partition_column_name RANGE [ LEFT | RIGHT ]
        FOR VALUES ( [ boundary_value [,...n] ] ) )

<data type> ::=
    datetimeoffset [ ( n ) ]
    | datetime2 [ ( n ) ]
    | datetime
    | smalldatetime
    | date
    | time [ ( n ) ]
    | float [ ( n ) ]
    | real [ ( n ) ]
    | decimal [ ( precision [ , scale ] ) ]
    | money
    | smallmoney
    | bigint
    | int
    | smallint
    | tinyint
    | bit
    | nvarchar [ ( n ) ]
    | nchar [ ( n ) ]
    | varchar [ ( n ) ]
    | char [ ( n ) ]
    | varbinary [ ( n ) ]
    | binary [ ( n ) ]

Arguments

database_name

The name of the database that will contain the new table. The default is the current database.

schema_name

The schema for the table. Specifying schema is optional. If blank, the users default schema will be used.

table_name

The name of the new table.

To create a local temporary table:

  1. Precede the table name with #.

  2. Specify the LOCATION = USER_DB option.

For more details on permitted table names, see Object Naming Rules (SQL Server PDW)

For more information about temporary tables, see tempdb Database (SQL Server PDW).

column_name

The name of a table column. For details on permitted column names, see Object Naming Rules (SQL Server PDW).

COLLATE Windows_collation_name

Specifies the collation for the expression. The collation must be one of the Windows collations supported by SQL Server. For a list of Windows collations supported by SQL Server, see Windows Collation Name (Transact-SQL).

The default collation is the appliance-level collation, Latin1_General_100_CI_AS_KS_WS. For more information about using collations in SQL Server PDW, see Collations (SQL Server PDW)

NULL | NOT NULL

Specifies whether NULL values are allowed in the column. The default is NULL.

[ CONSTRAINT constraint_name ] DEFAULT constant_expression

Specifies the default column value.

constraint_name

The optional name for the constraint. The constraint name is unique within the database. The name can be re-used in other databases.

constant_expression

The default value for the column. The expression must be a literal value or a a constant.

Constant expression examples - allowed:

  • ‘CA’

  • 4

Constant expression examples – not allowed:

  • ‘CA’ + ‘1’

  • 2+3

  • CURRENT_TIMESTAMP

<data_type>
datetimeoffset [ ( n ) ]

The default value for n is 7.

datetime2 [ ( n ) ]

Same as datetime, except that you can specify the number of fractional seconds. The default value for n is 7.

nvalue

Precision

Scale

0

19

0

1

21

1

2

22

2

3

23

3

4

24

4

5

25

5

6

26

6

7

27

7

datetime

Stores date and time of day with 19 to 23 characters according to the Gregorian calendar. The date can contain year, month, and day. The time contains hour, minutes, seconds.As an option, you can display three digits for fractional seconds. The storage size is 8 bytes.

smalldatetime

Stores a date and a time. Storage size is 4 bytes.

date

Stores a date using a maximum of 10 characters for year, month, and day according to the Gregorian calendar. The storage size is 3 bytes. Date is stored as an integer.

time [ ( n ) ]

The default value for n is 7.

float [ ( n ) ]

Approximate number data type for use with floating point numeric data. Floating point data is approximate, which means that not all values in the data type range can be represented exactly. n specifies the number of bits used to store the mantissa of the float number in scientific notation. Therefore, n dictates the precision and storage size. If n is specified, it must be a value between 1 and 53. The default value of n is 53.

nvalue

Precision

Storage size

1-24

7 digits

4 bytes

25-53

15 digits

8 bytes

SQL Server PDW treats n as one of two possible values. If 1<=n<=24, n is treated as 24. If 25<=n<=53, n is treated as 53.

The SQL Server PDW float data type complies with the ISO standard for all values of n from 1 through 53. The synonym for double precision is float(53).

real [ ( n ) ]

The definition of real is the same as float. The ISO synonym for real is float(24).

decimal [ ( precision [ , scale ] ) ]

Stores fixed precision and scale numbers.

precision

The maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point. The precision must be a value from 1 through the maximum precision of 38. The default precision is 18.

scale

The maximum number of decimal digits that can be stored to the right of the decimal point. scale must be a value from 0 through precision. Scale can only be specified if precision is specified. The default scale is 0; therefore, 0 <= scale <= precision. Maximum storage sizes vary, based on the precision.

Precision

Storage bytes

1 - 9

5

10-19

9

20-28

13

29-38

17

money | smallmoney

Data types that represent currency values.

Data Type

Storage bytes

money

8

smallmoney

4

bigint | int | smallint | tinyint

Exact-number data types that use integer data. The storage is shown in the following table.

Data Type

Storage bytes

bigint

8

int

4

smallint

2

tinyint

1

bit

An integer data type that can take the value of 1, 0, or NULL. SQL Server PDW optimizes storage of bit columns. If there are 8 or fewer bit columns in a table, the columns are stored as 1 byte. If there are from 9-16 bit columns, the columns are stored as 2 bytes, and so on.

nvarchar [ ( n ) ]

Variable-length Unicode character data. n can be a value from 1 through 4000. Storage size in bytes is two times the number of characters entered + 2 bytes. The data entered can be 0 characters in length.

nchar [ ( n ) ]

Fixed-length Unicode character data with a length of n characters. n must be a value from 1 through 4000. The storage size is two times n bytes.

varchar [ ( n ) ]

Variable-length, non-Unicode character data with a length of n bytes. n must be a value from 1 to 8000. The storage size is the actual length of data entered + 2 bytes.

char [ ( n ) ]

Fixed-length, non-Unicode character data with a length of n bytes. n must be a value from 1 to 8000. The storage size is n bytes. The default for n is 1.

varbinary [ ( n ) ]

Variable-length binary data. n can be a value from 1 to 8000. The storage size is the actual length of data entered + 2 bytes. The default value for n is 7.

binary [ ( n ) ]

Fixed-length binary data with a length of n bytes. n can be a value from 1 to 8000. The storage size is n bytes. The default value for n is 7.

For a table of data type conversions, see the Implicit Conversions section, of CAST and CONVERT (Transact-SQL).

AS SELECT select_criteria

Populates the new table with the results from a SELECT statement. select_criteria is the query that determines which data to copy to the new table. For information about SELECT statements, see SELECT (SQL Server PDW).

LOCATION = USER_DB

Specifies a user local temporary table. This option is required and used only when creating a temporary table. This option is not allowed when the table is not a temporary table.

CLUSTERED COLUMNSTORE INDEX

Creates an xVelocity memory-optimized clustered columnstore index on the table. The clustered columnstore index applies to all of the table data.

For more information, see Columnstore Tables in the General Remarks.

CLUSTERED INDEX ( index_column_name [,…n] )

Creates a rowstore clustered index on one or more key columns. This stores the data by row. Use index_column_name to specify the name of one or more key columns in the index.

For more information, see Rowstore Tables in the General Remarks.

DISTRIBUTION = { HASH ( distribution_column_name ) | REPLICATE }

Determines whether the table is a distributed table or a replicated table. If omitted, the table is replicated.

HASH ( distribution_column_name )

Creates a distributed table, in which the rows are spread across the Compute nodes and stored into individual parallel processing units called distributions. The HASH algorithm assigns each row to one distribution by hashing the value in distribution_column_name. In most cases, the distribution column should not be nullable. For more information about how to choose a distribution column, see Distributed and Replicated Tables (SQL Server PDW).

REPLICATE

Stores a copy of the table in full on each Compute node. Within each Compute node, the table is stored in a SQL Server filegroup that spans the Compute node..

PARTITION ( partition_column_name RANGE [ LEFT | RIGHT ] FOR VALUES ( [ boundary_value [,...n] ] ))

Creates one or more table partitions, which are horizontal table slices that allow you to perform operations on subsets of rows. Unlike the distribution column, table partitions do not determine the distribution where each row is stored. Instead, table partitions determine how the rows are grouped and stored within each distribution.

partition_column_name

Specifies the column that SQL Server PDW will use to partition the rows. This column can be any data type. SQL Server PDW sorts the partition column values in ascending order. The low-to-high ordering goes from LEFT to RIGHT for the purpose of the RANGE specification.

RANGE [ LEFT or RANGE RIGHT ]

Specifies whether the boundary value belongs to the partition on the left (lower values) or the partition on the right (higher values). The default is LEFT.

FOR VALUES( boundary_value [, …n)

Specifies the boundary values for the partition. If boundary_value is empty, the partitions map the whole table into one non-partitioned table. If applicable, a non-partitioned table can be split into two partitions at a later time.

boundary_value is a constant expression. It cannot be NULL. boundary_value must either match or be implicitly convertible to the data type in partition_column_name. boundary_value cannot be truncated during implicit conversion so that the size and scale of the value do not match the data type of partition_column_name.

See Example C. Creating a Partitioned Table in the Examples below.

Permissions

Requires CREATE TABLE permission on the database and ALTER SCHEMA permission on the schema that will contain the table. Or requires membership in the db_ddladmin fixed database role. Creating a partitioned table requires the ALTER ANY DATASPACE permission or membership in the db_ddladmin fixed database role.

The login that creates a local temporary table receives CONTROL, INSERT, SELECT and UPDATE permissions to the table.

General Remarks

Rowstore Tables

  • A table created without a clustered columnstore index is stored by row, and is called a rowstore table.

  • A rowstore table can have 1 clustered index and up to 999 nonclustered indexes at the same time.

  • Clustered and nonclustered indexes on a rowstore table can be dropped at any time.

  • You can change a rowstore table to a columnstore table by dropping all existing indexes on the table and creating a clustered columnstore index. For more information, see CREATE COLUMNSTORE INDEX (SQL Server PDW).

  • All rowstore tables are created with page compression. This is not user-configurable in SQL Server PDW.

Columnstore Tables

  • A columnstore table is a table that is stored by column.

  • A columnstore table has one clustered columnstore index and no other indexes.

  • To store a table by column, you can use CREATE TABLE to create the table with a clustered columnstore index, or you can use CREATE COLUMNSTORE INDEX to change an existing rowstore table to a columnstore table.

  • The columnstore index includes all columns in the table. These are displayed as included columns in the metadata. None of the columns are key columns.

  • You can change a columnstore table to a rowstore table by using CREATE INDEX (SQL Server PDW) with the DROP_EXISTING clause.

  • All columnstore tables are created with columnstore compression. This is not user-configurable in SQL Server PDW.

  • For a distributed table, the clustered columnstore index does not affect how the data is distributed; data is always distributed by row. The clustered columnstore index affects how the data is stored within each distribution.

For more information, see Clustered Columnstore Indexes (SQL Server PDW).

Local Temporary Tables are stored in the tempdb database. However, the LOCATION=USER_DB option is required when creating a temporary table. For more information about temporary tables, see. tempdb Database (SQL Server PDW).

For more information about how to choose a distribution column, see Distributed and Replicated Tables (SQL Server PDW).

For tables that are distributed and partitioned, each distribution contains all of the table partitions. For example, if there are eight distributions per Compute node and four table partitions, there will be 32 partitions per Compute node.

For table partitions, if you do not specify a boundary value, SQL Server PDW will create a partitioned table with one partition. If you specify one boundary value, the resulting table has two partitions — one for the values lower than the boundary value and one for the values higher than the boundary value. If you move a partition into a non-partitioned table, the non-partitioned table will receive the data, but will not have the partition boundaries in its metadata.

Limitations and Restrictions

Each table is a rowstore or a columnstore. Table data that is stored row-by-row is called a rowstore. Table data that is stored column-by-column is called a columnstore. Mixing the two forms of data storage is not allowed.

You cannot define a DEFAULT constraint on a distribution column.

When using partitions, the partition column cannot have a Unicode-only collation. For example, the following statement fails.

CREATE TABLE t1 ( c1 varchar(20) COLLATE Divehi_90_CI_AS_KS_WS) WITH (PARTITION (c1 RANGE FOR VALUES (N'')))

Global temporary tables that begin with ## are not supported.

Local Temporary Tables have the following limitations and restrictions:

  • They are visible only to the current session. SQL Server PDW drops them automatically at the end of the session. They can be dropped explicitly with the DROP TABLE statement.

  • They cannot be renamed.

  • They cannot have partitions or views.

  • Their permissions cannot be changed. GRANT, DENY, and REVOKE statements cannot be used with local temporary tables.

  • Database console commands are blocked for temporary tables.

  • If more than one local temporary table is used within a batch, each must have a unique name. If multiple sessions are running the same batch and creating the same local temporary table, SQL Server PDW internally appends a numeric suffix to the local temporary table name to maintain a unique name for each local temporary table.

For information on minimum and maximum constraints on tables, see Minimum and Maximum Values (SQL Server PDW).

If boundary_value is a literal value that must be implicitly converted to the data type in partition_column_name, a discrepancy will occur. The literal value is displayed through the SQL Server PDW system views, but the converted value is used for SQL operations.

Locking

Takes an exclusive lock on the table. Takes a shared lock on the DATABASE, SCHEMA, and SCHEMARESOLUTION objects.

Examples

A. Create a replicated table

The following example creates a replicated table with three columns and without partitions. The id column includes a NOT NULL constraint. The table is created with a CLUSTERED COLUMNSTORE INDEX, which gives better performance better data compression over a heap or rowstore clustered index.

Copy image Copy Code
CREATE TABLE myTable
  (
    id int NOT NULL,
    lastName varchar(20),
    zipCode varchar(6)
  )
WITH ( CLUSTERED COLUMNSTORE INDEX );

B. Create a hash-distributed table

The following example creates the same table as the previous example. However, this table is distributed (on the id column) instead of replicated. The table is created with a CLUSTERED COLUMNSTORE INDEX, which gives better performance better data compression over a heap or rowstore clustered index.

Copy image Copy Code
CREATE TABLE myTable
  (
    id int NOT NULL,
    lastName varchar(20),
    zipCode varchar(6)
  )
WITH
  (
    DISTRIBUTION = HASH (id),
    CLUSTERED COLUMNSTORE INDEX
  );

C. Create a local temporary table

The following creates a local temporary table named #myTable in AdventureWorkdsDW2008. The table is specified with a 3-part name. The temporary table name starts with a #, and the required LOCATION=USER_DB option is provided.

Both distributed and replicated tables are allowed for temporary tables. This example shows how to create a distributed temporary table.

Copy image Copy Code
CREATE TABLE
    AdventureWorksPDW2012
  .dbo.#myTable
  (
   id int NOT NULL,
   lastName varchar(20),
   zipCode varchar(6)
  )
WITH
  (
    LOCATION = USER_DB,
    DISTRIBUTION = HASH (id),
    CLUSTERED COLUMNSTORE INDEX
  )
;

D. Creating a partitioned table

The following example creates the same table as shown in example A, with the addition of RANGE LEFT partitioning on the id column. It specifies four partition boundary values, which results in five partitions.

Copy image Copy Code
CREATE TABLE myTable
  (
    id int NOT NULL,
    lastName varchar(20),
    zipCode int)
WITH
  (

    PARTITION ( id RANGE LEFT FOR VALUES (10, 20, 30, 40 )),
    CLUSTERED COLUMNSTORE INDEX
  )
;

In this example, data will be sorted into the following partitions:

  • Partition 1: col <= 10

  • Partition 2: 10 < col <= 20

  • Partition 3: 20 < col <= 30

  • Partition 4: 30 < col <= 40

  • Partition 5: 40 < col

If this same table was partitioned RANGE RIGHT instead of RANGE LEFT (default), the data will be sorted into the following partitions:

  • Partition 1: col < 10

  • Partition 2: 10 <= col < 20

  • Partition 3: 20 <= col < 30

  • Partition 4: 30 <= col < 40

  • Partition 5: 40 <= col

E. Create a partitioned table with one partition

The following example creates a partitioned table with one partition. It does not specify any boundary values, which results in one partition.

Copy image Copy Code
CREATE TABLE myTable (
    id int NOT NULL,
    lastName varchar(20),
    zipCode int)
WITH
    (
      PARTITION ( id RANGE LEFT FOR VALUES ( )),
      CLUSTERED COLUMNSTORE INDEX
    )
;

F. Create a table with date partitioning

The following example creates a new table named myTable, with partitioning on a date column. By using RANGE RIGHT and dates for the boundary values, it puts a month of data in each partition.

Copy image Copy Code
CREATE TABLE myTable (
    l_orderkey      bigint,
    l_partkey       bigint,
    l_suppkey       bigint,
    l_linenumber    bigint,
    l_quantity      decimal(15,2),
    l_extendedprice decimal(15,2),
    l_discount      decimal(15,2),
    l_tax           decimal(15,2),
    l_returnflag    char(1),
    l_linestatus    char(1),
    l_shipdate      date,
    l_commitdate    date,
    l_receiptdate   date,
    l_shipinstruct  char(25),
    l_shipmode      char(10),
    l_comment       varchar(44))
WITH
  (
    DISTRIBUTION = HASH (l_orderkey),
    CLUSTERED COLUMNSTORE INDEX,
    PARTITION ( l_shipdate
    RANGE RIGHT FOR VALUES
      (
      '1992-01-01','1992-02-01','1992-03-01','1992-04-01','1992-05-01','1992-06-01','1992-07-01','1992-08-01','1992-09-01','1992-10-01','1992-11-01','1992-12-01',
      '1993-01-01','1993-02-01','1993-03-01','1993-04-01','1993-05-01','1993-06-01','1993-07-01','1993-08-01','1993-09-01','1993-10-01','1993-11-01','1993-12-01',
      '1994-01-01','1994-02-01','1994-03-01','1994-04-01','1994-05-01','1994-06-01','1994-07-01','1994-08-01','1994-09-01','1994-10-01','1994-11-01','1994-12-01'
    ))
  );

G. Specify a column collation for a table

In the following example, the table MyTable is created with two different column collations. By default, the column, mycolumn1, has the appliance-level collation Latin1_General_100_CI_AS_KS_WS. The column, mycolumn2 has the collation Frisian_100_CS_AS.

Copy image Copy Code
CREATE TABLE MyTable
  (
    mycolumnnn1 nvarchar,
    mycolumn2 nvarchar COLLATE Frisian_100_CS_AS )
WITH ( CLUSTERED COLUMNSTORE INDEX )
;

H. Specify a DEFAULT constraint for a column

The following example shows the syntax to specify a default value for a column. The colA column has a default constraint named constraint_colA and a default value of 0.

Copy image Copy Code
CREATE TABLE MyTable
  (
    colA int CONSTRAINT constraint_colA DEFAULT 0,
    colB nvarchar COLLATE Frisian_100_CS_AS
  )
WITH ( CLUSTERED COLUMNSTORE INDEX )
;

I. Create a table with a clustered columnstore index

The following example creates a distributed table with a clustered columnstore index. Each distribution will be stored as a columnstore.

The clustered columnstore index does not affect how the data is distributed; data is always distributed by row. The clustered columnstore index affects how the data is stored within each distribution.

Copy image Copy Code
CREATE TABLE MyTable
  (
    colA int CONSTRAINT constraint_colA DEFAULT 0,
    colB nvarchar COLLATE Frisian_100_CS_AS
  )
WITH
  (
    DISTRIBUTION = HASH ( colB ),
    CLUSTERED COLUMNSTORE INDEX
  )
;

See Also

Reference

CREATE TABLE AS SELECT (SQL Server PDW)
DROP TABLE (SQL Server PDW)
ALTER TABLE (SQL Server PDW)


CREATE TABLE AS SELECT (SQL Server PDW)

[image]

CREATE TABLE AS SELECT (SQL Server PDW)
See Also Send Feedback

Creates a new table in SQL Server PDW that is populated with the results from a SELECT statement. Use this statement to copy data from one or more existing tables to a new table.

For example, use the Create Table as Select (CTAS) statement to:

  • Redistribute a SQL Server PDW distributed table using a different distribution column.

  • Create an xVelocity memory optimized clustered columnstore index on a subset of the columns in the table.

  • Import Hadoop data from an external table into SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE TABLE [ database_name . [ schema_name ] . | schema_name. ] table_name
        [ ( { column_name } [ ,...n ] ) ]
    WITH (
        DISTRIBUTION = { HASH( distribution_column_name ) | REPLICATE }
            [ , <CTAS_table_option> [ ,...n ] ]
    )
    AS <select_statement>
[;]

<CTAS_table_option> ::=
    LOCATION = USER_DB
    | CLUSTERED COLUMNSTORE INDEX
    | CLUSTERED INDEX ( { index_column_name [ ASC | DESC ] } [ ,...n ] )
    | PARTITION( partition_column_name RANGE [ LEFT | RIGHT
        FOR VALUES ( [ boundary_value [,...n] ] ) ) ]

<select_statement> ::=
    [ WITH <common_table_expression> [ ,...n ] ]
    SELECT <select_criteria>

Arguments

database_name

The name of the database that will contain the new table. The default is the current database.

schema_name

The schema of the new table. Specifying schema is optional.

table_name

The name of the new table.

To create a local temporary table:

  1. Specify the table name with 3-part naming.

  2. Precede the table name with #.

  3. Specify the LOCATION = USER_DB option.

For more details on permitted table names, see Object Naming Rules (SQL Server PDW)

column_name [, …]

The list of one or more column names for the new table. For details on permitted column names, see Object Naming Rules (SQL Server PDW).

The column list is optional for the CTAS statement. When columns are listed, the column names do not need to match the column names in the select results. However, the number of columns in the column list must match the number of columns in the select results. You cannot specify NULL | NOT NULL for the columns in the CTAS statement; the nullability property is derived from the columns and expressions in the SELECT results.

LOCATION = USER_DB

Table is located in the user database. This option is required and used only when creating a temporary table. This option is not allowed when the table is not a temporary table.

CLUSTERED COLUMNSTORE INDEX

Creates an xVelocity memory optimized columnstore index. The clustered columnstore index includes all columns of the table and stores the data by column. A table that has a clustered columnstore index cannot have any other clustered or nonclustered indexes.

For best performance and data compression, we recommend using the clustered columnstore index instead of the rowstore clustered index.

CLUSTERED INDEX ( index_column_name [,…n] )

Creates a clustered rowstore index on one or more columns, called key columns. This stores the data by row. Use index_column_name to specify the name of one or more key columns in the index..

For best performance and data compression, we recommend using the clustered columnstore index instead of this rowstore clustered index.

DISTRIBUTION = { HASH ( distribution_column_name ) | REPLICATE }

The physical data storage method. This option is required for the CTAS statement.

HASH ( distribution_column_name )

Distributes table rows across the Compute nodes. Assigns each row to one distribution by hashing the value in distribution_column_name. The nullability property is derived from the columns and expressions in the SELECT results. For more information about how to choose a distribution column, see Distributed and Replicated Tables (SQL Server PDW).

REPLICATE

Stores each table in full once per Compute node.

PARTITION ( partition_column_name RANGE [ LEFT | RIGHT ] FOR VALUES ( [ boundary_value [,...n] ] ))

Creates one or more table partitions, which are horizontal table slices that allow you to perform operations on subsets of rows. Unlike the distribution column, table partitions do not determine the distribution where each row is stored. Instead, table partitions determine how the rows are grouped and stored within each distribution.

partition_column_name

Specifies the column to partition the rows on. This column can be any data type. SQL Server PDW sorts the partition column values in ascending order. The low-to-high ordering goes from LEFT to RIGHT for the purpose of the RANGE specification.

RANGE [ LEFT or RANGE RIGHT ]

Specifies whether the boundary value belongs to the partition on the left (lower values) or the partition on the right (higher values). The default is LEFT.

FOR VALUES( boundary_value [, …n)

Specifies the boundary values for the partition. If boundary_value is empty, the partitions map the whole table into one non-partitioned table. If applicable, a non-partitioned table can be split into two partitions at a later time.

boundary_value is a constant expression. It cannot be NULL. boundary_value must either match or be implicitly convertible to the data type in partition_column_name. boundary_value cannot be truncated during implicit conversion so that the size and scale of the value do not match the data type of partition_column_name.

WITH common_table_expression

Specifies a temporary named result set, known as a common table expression (CTE). For more information, see WITH common_table_expression (SQL Server PDW).

SELECT <select_criteria>

Populates the new table with the results from a SELECT statement. select_criteria is the body of the SELECT statement that determines which data to copy to the new table. For information about SELECT statements, see SELECT (SQL Server PDW).

Permissions

Requires CREATE TABLE permission on the database and ALTER SCHEMA permission on the schema that will contain the new table. Or requires membership in the db_ddladmin fixed database role. Also requires SELECT permission on any objects referenced in the select_criteria.

Requires CREATE permission on the database.

General Remarks

For more information about creating tables and temporary tables, see CREATE TABLE (SQL Server PDW)

The CTAS statement creates a non-partitioned table by default, even if the source table is partitioned. To create a partitioned table with the CTAS statement, you must specify the partition option.

Limitations and Restrictions

SET ROWCOUNT (SQL Server PDW) has no effect on this statement. To achieve a similar behavior, use TOP (SQL Server PDW).

Locking

Takes an exclusive lock on the table. Takes a shared lock on the DATABASE, SCHEMA, and SCHEMARESOLUTION objects.

Examples

A. Create a table by using CREATE TABLE AS SELECT (CTAS)

The following example creates a new table named myTable, using the column definitions and data from the source table dimCustomer.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE TABLE myTable
WITH
  (
    CLUSTERED COLUMNSTORE INDEX,
    DISTRIBUTION = HASH (CustomerKey)
  )
AS SELECT * FROM dimCustomer;

B. Create a table by using CREATE TABLE AS SELECT (CTAS) and Aliasing

The following example creates a table named myTable (c, ln). It uses the column aliases in the SELECT statement for the names of the columns in the new table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE TABLE myTable
WITH
  (
    CLUSTERED COLUMNSTORE INDEX,
    DISTRIBUTION = HASH (c)
  )
AS SELECT CustomerKey AS c, LastName AS ln
    FROM dimCustomer;

C. Use a Query Hint with CREATE TABLE AS SELECT (CTAS)

This query shows the basic syntax for using a query join hint with the CTAS statement. After the query is submitted to the Control node, SQL Server, running on the Compute nodes, will apply the hash join strategy when generating the SQL Server query plan. For more information on join hints and how to use the OPTION clause, see OPTION (SQL Server PDW).

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE TABLE dbo.FactInternetSalesNew
WITH
  (
    CLUSTERED COLUMNSTORE INDEX,
    DISTRIBUTION = REPLICATE
  )
AS SELECT T1.* FROM dbo.FactInternetSales T1 JOIN dbo.DimCustomer T2
ON ( T1.CustomerKey = T2.CustomerKey )
OPTION ( HASH JOIN );

D. Use CREATE TABLE AS SELECT (CTAS) to change collations

This example uses CTAS to change the collation of all the columns in the DatabaseLog table to Serbian_Cyrillic_100_BIN. The example puts the results of the CTAS into the DatabaseLog2 table

Copy image Copy Code
/* CTAS the DatabaseLog table to DatabaseLog2 table with different collation */
USE
    AdventureWorksPDW2012
  ;
If EXISTS (SELECT * from sys.tables t where t.name = 'DatabaseLog2')
DROP TABLE [DatabaseLog2];

CREATE TABLE DatabaseLog2
WITH
  (
    CLUSTERED COLUMNSTORE INDEX,
    DISTRIBUTION = REPLICATE
  )
AS SELECT
    [DatabaseUser] COLLATE Serbian_Cyrillic_100_BIN AS [DatabaseUser],
    [Event] COLLATE Serbian_Cyrillic_100_BIN AS [Event],
    [Schema] COLLATE Serbian_Cyrillic_100_BIN AS [Schema],
    [Object] COLLATE Serbian_Cyrillic_100_BIN AS [Object],
    [TSQL] COLLATE Serbian_Cyrillic_100_BIN AS [TSQL]
FROM DatabaseLog;

E. Use CREATE TABLE AS SELECT to import Hadoop data from an external table

To import data from an external table, simply use CREATE TABLE AS SELECT to select from the external table. The syntax to select data from an external table into SQL Server PDW is the same as the syntax for selecting data from a distributed or replicated table.

The following example defines an external table on a Hadoop cluster. It then uses CREATE TABLE AS SELECT to select from the external table. This imports the data from Hadoop text-delimited files and stores the data into a new SQL Server PDW table.

Copy image Copy Code
--Create the external table called ClickStream.
CREATE EXTERNAL TABLE ClickStreamExt (
    url varchar(50),
    event_date date,
    user_IP varchar(50)
)
WITH (
    LOCATION = 'hdfs://MyHadoop:5000/tpch1GB/employee.tbl',
    FORMAT_OPTIONS ( FIELD_TERMINATOR = '|')
)
;

--Use your own processes to create the Hadoop text-delimited files on the Hadoop Cluster.

--Use CREATE TABLE AS SELECT to import the Hadoop data into a new
--SQL Server PDW table called ClickStreamPDW
CREATE TABLE ClickStreamPDW
WITH
  (
    CLUSTERED COLUMNSTORE INDEX,
    DISTRIBUTION = HASH (user_IP)
  )
AS SELECT * FROM ClickStreamExt
;

See Also

Reference

CREATE TABLE (SQL Server PDW)
CREATE EXTERNAL TABLE (SQL Server PDW)
CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW)
DROP TABLE (SQL Server PDW)
ALTER TABLE (SQL Server PDW)


CREATE VIEW (SQL Server PDW)

[image]

CREATE VIEW (SQL Server PDW)
See Also Send Feedback

Creates a view in SQL Server PDW. A view is a virtual table that contains the results of a query. The query, and not the data, is stored in the database.

  • You can query a view by using the same SQL statements that you would use to query a table.

  • By setting permissions on views, you can allow specific users to read or modify the table data that is in the view.

  • A view does not have to be a simple subset of the rows and columns of one table. You can create a view based on a select statement that queries multiple tables or views from multiple databases.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE VIEW [ schema_name . ] view_name [  ( column_name [ ,...n ] ) ]
AS <select_statement>
[;]

select_statement> ::=
    [ WITH <common_table_expression> [ ,...n ] ]
    SELECT <select_criteria>

Arguments

[ schema_name . ] view_name

The two-part name of the view. View names must follow the rules for identifiers. See Object Naming Rules (SQL Server PDW) for full details on permitted view names.

column_name [,...n]

One or more column names for the view.

If column names are not specified, the view columns will use the column names specified in the SELECT statement.

common_table_expression

Specifies a temporary named result set, known as a common table expression (CTE). For more information, see WITH common_table_expression (SQL Server PDW).

SELECT <select_criteria>

The SELECT statement that defines the view. For more information about SELECT statements, see SELECT (SQL Server PDW).

The SELECT statement can contain almost everything that is supported for the SELECT statement. This includes rowstore and columnstore tables, functions, and multiple SELECT statements that are separated by UNION or UNION ALL.

The SELECT statement cannot contain:

  • An ORDER BY clause, unless there is also a TOP clause in the SELECT statement. The ORDER BY clause is used only to determine the rows that are returned by the TOP clause in the view definition. The ORDER BY clause does not guarantee ordered results when the view is queried, unless ORDER BY is also specified in the query itself. See TOP (SQL Server PDW) for information on the TOP clause.

  • The INTO keyword.

  • An OPTION clause that contains a query hint.

Permissions

Requires CREATE VIEW permission on the database and ALTER SCHEMA permission on the schema that will contain the view. Or requires membership in the db_ddladmin fixed database role.

Limitations and Restrictions

For information on minimum and maximum constraints on views, see Minimum and Maximum Values (SQL Server PDW).

Locking

Takes an exclusive lock on the VIEW. Takes a shared lock on the DATABASE, SCHEMA, and SCHEMARESOLUTION objects.

Metadata

To see the Data Definition Language (DDL) for a view, you can use the sys.views metadata table.

Examples

A. Creating a simple view

The following example creates a view by selecting only some of the columns from the source table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE VIEW DimEmployeeBirthDates AS
SELECT FirstName, LastName, BirthDate
FROM DimEmployee;

B. Create a view by joining two tables

The following example creates a view by using a SELECT statement with an OUTER JOIN. The results of the join query populate the view.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE VIEW view1
AS SELECT fis.CustomerKey, fis.ProductKey, fis.OrderDateKey, fis.SalesTerritoryKey, dst.SalesTerritoryRegion
FROM FactInternetSales AS fis
LEFT OUTER JOIN DimSalesTerritory AS dst
ON (fis.SalesTerritoryKey=dst.SalesTerritoryKey);

See Also

Reference

SELECT (SQL Server PDW)
DROP VIEW (SQL Server PDW)


DROP DATABASE (SQL Server PDW)

[image]

DROP DATABASE (SQL Server PDW)
See Also Send Feedback

Removes a user database in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP DATABASE database_name
[;]

Arguments

database_name

The name of the database to be removed. To display a list of databases on the appliance, use sys.databases (SQL Server PDW).

Permissions

Requires the CONTROL permission on the database, or ALTER ANY DATABASE permission, or membership in the db_owner fixed database role.

Error Handling

With regard to differences between SQL Server PDW and SQL Server, these products issue different errors when a user attempts to drop a database in use by the current session, and the user does not have the required permissions to drop the database. In this situation, SQL Server PDW issues error 3702 (in use) and SMP SQL Server issues 3701 (no permission).

General Remarks

Dropping a database removes the database from an instance of SQL Server PDW and deletes the files used by the database.

Limitations and Restrictions

You cannot drop a database that is open for reading or writing by any user, including the USE (DW-SQL) statement.

The DROP DATABASE statement cannot be run from within an explicit transaction. For information on explicit transactions, see Transactions (SQL Server PDW).

You cannot drop the system databases master and tempdb.

Locking

Takes a shared lock on the DATABASE object.

Examples

The following example removes the Sales database.

Copy image Copy Code
DROP DATABASE Sales;

See Also

Reference

CREATE DATABASE (SQL Server PDW)
ALTER DATABASE (SQL Server PDW)


DROP EXTERNAL DATA SOURCE (SQL Server PDW)

[image]

DROP EXTERNAL DATA SOURCE (SQL Server PDW)
See Also Send Feedback

Removes a Hadoop external data source from SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Drop an external data source
DROP EXTERNAL DATA SOURCE external_data_source_name
[;]

Arguments

external_data_source_name

The name of the external data source to drop.

Metadata

To view a list of external data sources use the sys.external_data_sources system view.

Copy image Copy Code
SELECT * FROM sys.external_data_sources;

Permissions

Requires CONTROL SERVER permission or ALTER ANY EXTERNAL DATA SOURCE

Locking

Takes a shared lock on the EXTERNALDATASOURCE object.

General Remarks

Dropping an external data source does not remove the external data.

Examples

A. Using basic syntax

Copy image Copy Code
DROP EXTERNAL DATA SOURCE mydatasource;

See Also


DROP EXTERNAL FILE FORMAT (SQL Server PDW)

[image]

DROP EXTERNAL FILE FORMAT (SQL Server PDW)
See Also Send Feedback

Removes a Hadoop external file format from SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

-- Drop an external file format
DROP EXTERNAL FILE FORMAT external_file_format_name
[;]

Arguments

external_file_format_name

The name of the external file format to drop.

Metadata

To view a list of external file formats use the sys.external_file_formats (SQL Server PDW) system view.

Copy image Copy Code
SELECT * FROM sys.external_file_formats;

Permissions

Requires CONTROL SERVER permission or ALTER ANY EXTERNAL FILE FORMAT

General Remarks

Dropping an external file format does not remove the external data.

Locking

Takes a shared lock on the EXTERNALFILEFORMAT object.

Examples

A. Using basic syntax

Copy image Copy Code
DROP EXTERNAL FILE FORMAT myfileformat;

See Also


DROP EXTERNAL TABLE (SQL Server PDW)

[image]

DROP EXTERNAL TABLE (SQL Server PDW)
See Also Send Feedback

Removes an external table from SQL Server PDW. This does not delete the external data.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP EXTERNAL TABLE [ database_name . [schema_name ] . | schema_name . ]table_name
[;]

Arguments

[ database_name . [schema_name] . | schema_name . ] table_name

The one- to three-part name of the external table to remove. The table name can optionally include the schema, or the database and schema.

Permissions

  • Requires CONTROL permission on the table, or ALTER permission on the schema to which the table belongs, or membership in the db_ddladmin fixed database role.

  • Requires ADMINISTER BULK OPERATIONS permission.

General Remarks

Dropping an external table removes all table-related metadata from SQL Server PDW. It does not delete the external data.

Examples

A. Using basic syntax

Copy image Copy Code
DROP EXTERNAL TABLE SalesPerson;
DROP EXTERNAL TABLE dbo.SalesPerson;
DROP EXTERNAL TABLE EasternDivision.dbo.SalesPerson;

B. Dropping an external table from the current database

The following example removes the ProductVendor1 table, its data, indexes, and any dependent views from the current database.

Copy image Copy Code
DROP EXTERNAL TABLE ProductVendor1;

C. Dropping a table from another database

The following example drops the SalesPerson table in the EasternDivision database.

Copy image Copy Code
DROP EXTERNAL TABLE EasternDivision.dbo.SalesPerson;

See Also

Reference

DROP TABLE (SQL Server PDW)
CREATE TABLE (SQL Server PDW)
ALTER TABLE (SQL Server PDW)


DROP INDEX (SQL Server PDW)

[image]

DROP INDEX (SQL Server PDW)
See Also Send Feedback

Removes and deletes an index in SQL Server PDW.

To drop an index you can also use CREATE INDEX (SQL Server PDW) with the DROP_EXISTING clause. This converts an existing index to an index of a different type and keeps the same name.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP INDEX index_name ON [ database_name . [schema_name ] . | schema_name . ] table_name
[;]

Arguments

index_name

The name of the index to be dropped. The index can be of any type including a rowstore clustered index, a rowstore non-clustered index, or a clustered columnstore index.

[ database_name . [schema_name ] . | schema_name . ] table_name

The three-part name of the table that has the index to drop. The table name can optionally include the schema or the database and schema.

Permissions

Requires ALTER permission on the table or membership in one of the following roles:

  • sysadmin fixed server role.

  • db_ddladmin fixed database role.

  • db_owner fixed database role.

General Remarks

When an index is dropped, SQL Server PDW removes the index and deletes all metadata for the index. Table values are not deleted.

When a clustered columnstore index is dropped:

  • The columnstore table is converted into a rowstore heap.

  • All auto-created statistics are dropped.

  • Statistics created with the CREATE STATISTICS statement are not dropped.

For replicated tables, dropping a clustered index, both rowstore and columnstore, runs in parallel across the Compute nodes. If the table is distributed, dropping an index runs in parallel across the Compute nodes and across the distributions within each Compute node.

Limitations and Restrictions

Each table can have only one DROP INDEX command running. Additional DROP INDEX requests will wait until the current request finished.

DROP INDEX cannot run within a transaction.

Locking

Takes an exclusive lock on the TABLE object. Takes a shared lock on the DATABASE object.

Examples

A. Dropping an index on a table

The following example deletes the index VendorIDIndex on the ProductVendor table.

Copy image Copy Code
DROP INDEX VendorIDIndex ON ProductVendor;

See Also

Reference

CREATE INDEX (SQL Server PDW)
ALTER INDEX (SQL Server PDW)


DROP PROCEDURE (SQL Server PDW)

[image]

DROP PROCEDURE (SQL Server PDW)
See Also Send Feedback

Drops a stored procedure on a SQL Server PDW database.

Syntax

Visual Basic
DROP { PROC | PROCEDURE } { [ schema_name. ] procedure_name }

Arguments

schema_name

Is the name of the schema to which the procedure belongs. A server name or database name cannot be specified.

procedure_name

Is name of the stored procedure to be removed. Procedure names must follow the rules for identifiers.

Permissions

Requires CONTROL permission on the procedure, or ALTER permission on the schema to which the procedure belongs, or membership in the db_ddladmin fixed server role.

Locking

Takes an exclusive lock on the procedure. Takes a shared lock on the DATABASE, SCHEMA, and SCHEMARESOLUTION object.

Examples

The following example removes the dbo.uspMyProc stored procedure in the current database.

Copy image Copy Code
DROP PROCEDURE dbo.uspMyProc;
GO

See Also

Reference

CREATE PROCEDURE (SQL Server PDW)
ALTER PROCEDURE (SQL Server PDW)


DROP STATISTICS (SQL Server PDW)

[image]

DROP STATISTICS (SQL Server PDW)
See Also Send Feedback

Removes and deletes query optimization statistics from a SQL Server PDW table, including an external table.

For more information, see Understanding Query Optimization Statistics (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP STATISTICS [ database_name . [ schema_name ] . | schema_name . ] table_name.statistics_name
[;]

Arguments

[ database_name . [ schema_name] . | schema_name . ] table_name

The one-, two-, or three-part name of the table that contains the statistics to drop.

statistics_name

The name of the statistics to drop.

Permissions

Requires ALTER permission on the table.

Performance

Dropping statistics can affect the query execution plan chosen by the query optimizer. This could have a negative impact on query performance.

Examples

A. Dropping statistics from a table

The following examples drop the CustomerStats1 statistics from table Customer.

Copy image Copy Code
DROP STATISTICS Customer.CustomerStats1;
DROP STATISTICS dbo.Customer.CustomerStats1;

See Also

Reference

CREATE STATISTICS (SQL Server PDW)
UPDATE STATISTICS (SQL Server PDW)


DROP TABLE (SQL Server PDW)

[image]

DROP TABLE (SQL Server PDW)
See Also Send Feedback

Removes and deletes a user-defined table from SQL Server PDW.

To drop an external table, see DROP EXTERNAL TABLE (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP TABLE [ database_name . [ schema_name ] . | schema_name . ]table_name
[;]

Arguments

[ database_name . [schema_name ] . | schema_name . ] table_name

The one- to three-part name of the table to remove and delete. The table name can optionally include the schema, or the database and schema.

Permissions

Requires CONTROL permission on the table, ALTER permission on the schema to which the table belongs, or membership in the db_ddladmin fixed database role. To drop a partitioned table also requires ALTER ANY DATASPACE permission or membership in the db_ddladmin fixed database role.

General Remarks

Dropping a table removes all table-related data, including the table definition, indexes, row data, constraints, and table permissions.

Limitations and Restrictions

Any views dependent on the table will be broken after the table is dropped.

Locking

Takes an exclusive lock on the table. Takes a shared lock on the DATABASE, and SCHEMA objects.

Examples

A. Using basic syntax

Copy image Copy Code
DROP TABLE SalesPerson;
DROP TABLE dbo.SalesPerson;
DROP TABLE EasternDivision.dbo.SalesPerson;

B. Dropping a table from the current database

The following example removes the ProductVendor1 table, its data, indexes, and any dependent views from the current database.

Copy image Copy Code
DROP TABLE ProductVendor1;

C. Dropping a table from another database

The following example drops the SalesPerson table in the EasternDivision database.

Copy image Copy Code
DROP TABLE EasternDivision.dbo.SalesPerson;

See Also

Reference

DROP EXTERNAL TABLE (SQL Server PDW)
CREATE TABLE (SQL Server PDW)
ALTER TABLE (SQL Server PDW)


DROP SCHEMA (SQL Server PDW)

[image]

DROP SCHEMA (SQL Server PDW)
See Also Send Feedback

Removes a schema from the database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP SCHEMA schema_name

Arguments

schema_name

Is the name by which the schema is known within the database.

Remarks

The schema that is being dropped must not contain any objects. If the schema contains objects, the DROP statement fails.

Information about schemas is visible in the sys.schemas catalog view.

Permissions

Requires CONTROL permission on the schema or ALTER ANY SCHEMA permission on the database.

Locking

Takes an exclusive lock on the SCHEMA and on the SCHEMARESOLUTION object. Takes a shared lock on the DATABASE object.

Examples

The following example creates the schema Sprockets and a table Sprockets.NineProngs.

Copy image Copy Code
USE AdventureWorks2012;
GO
CREATE SCHEMA Sprockets;
CREATE TABLE NineProngs (source int, cost int, partnumber int);
GO

The following statements drop the schema. Note that you must first drop the table that is contained by the schema.

Copy image Copy Code
DROP TABLE Sprockets.NineProngs;
DROP SCHEMA Sprockets;
GO

See Also

Reference

CREATE SCHEMA (SQL Server PDW)
ALTER SCHEMA (SQL Server PDW)
sys.schemas (SQL Server PDW)
Functions (SQL Server PDW)


DROP VIEW (SQL Server PDW)

[image]

DROP VIEW (SQL Server PDW)
See Also Send Feedback

Deletes a view from the current SQL Server PDW database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP VIEW [ schema_name . ] view_name
[;]

Arguments

[ schema_name . ] view_name

The two-part name for the view to be removed.

Permissions

Requires CONTROL permission on the view, ALTER permission on the schema containing the view, or membership in the db_ddladmin fixed server role.

General Remarks

When you drop a view, the view is removed from the system catalog. All permissions and all dependent views are dropped.

Metadata

To see the Data Definition Language (DDL) for a view, you can use the sys.views catalog view.

Locking

Takes an exclusive lock on the VIEW. Takes a shared lock on the DATABASE, SCHEMA, and SCHEMARESOLUTION objects.

Examples

A. Drop a view

The following example removes the Reorder view from the local database.

Copy image Copy Code
DROP VIEW Reorder;

See Also

Reference

CREATE VIEW (SQL Server PDW)


RENAME (SQL Server PDW)

[image]

RENAME (SQL Server PDW)
See Also Send Feedback

Renames a user-created table or database in SQL Server PDW.

Syntax

RENAME <class_type> [ :: ] entity_name TO new_entity_name
[;]

<class_type> ::=
{
    DATABASE
    | OBJECT
}

Arguments

entity_name

The name of the table or database to be renamed. For tables, you can specify entity_name as a three-part table name. The table can any type of user-created table including distributed, replicated, or external. You cannot rename a database to any of these Reserved Database Names (SQL Server PDW).

new_entity_name

The new name of the table or database. For tables, you can specify entity_name as a two-part table name.

DATABASE

Specifies that the entity to be renamed is a database.

OBJECT

Specifies that the entity to be renamed is an object. Tables are the only objects supported in this release.

Permissions

Requires the ALTER permission on the table.

General Remarks

When renaming a table, all objects and properties associated with the table are updated to reference the new table name. For example, table definitions, indexes, constraints, and permissions are updated. Views are not updated. For more information, see Limitations and Restrictions.

RENAME cannot be used to move a table to a different schema; use ALTER SCHEMA … TRANSFER instead.

Renaming an external table changes the table name within SQL Server PDW. It does not effect the location of the external data for the table.

Limitations and Restrictions

You cannot rename an entity that is in use by another user.

You cannot rename indexes and views. To accomplish this task, you can drop the index or view and then recreate it with the new name.

When renaming a database, views are not updated. Each view, inside or outside of the database, that uses the former database name will become invalid. For example, if the Sales database is renamed, a view that contains SELECT * FROM Sales.dbo.table1 will become invalid. To resolve this, you can either avoid using three-part names in views, or update the views to reference the new database name.

When renaming a table, views are not updated to reference the new table name. Each view, inside or outside of the database, that references the former table name will become invalid. To resolve this, you can update each view to reference the new table name.

Locking

Takes a shared lock on the DATABASE object, and an exclusive lock on the object being renamed. Renaming an object takes a shared lock on the SCHEMA object.

Examples

RENAME DATABASE::Sales TO SalesArchive;

RENAME DATABASE Sales TO SalesArchive;

RENAME OBJECT::Customer TO NewCustomer;

RENAME OBJECT Customer TO NewCustomer;

See Also


RESTORE DATABASE (SQL Server PDW)

[image]

RESTORE DATABASE (SQL Server PDW)
See Also Send Feedback

Restores a SQL Server PDW user database from a database backup to a SQL Server PDW appliance. The database is restored from a backup that was previously created by the SQL Server PDW BACKUP DATABASE (SQL Server PDW) command. Use the backup and restore operations to build a disaster recovery plan, or to move databases from one appliance to another.

Note Note

Restoring master includes restoring appliance login information. To restore master, use the Restore the Master Database (Analytics Platform System) page in the Configuration Manager tool. An administrator with access to the Control node can perform this operation.

For more information about SQL Server PDW database backups, see Backup and Restore (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

Restore the master database
-- Use the Configuration Manager tool.

Restore a full user database backup.
RESTORE DATABASE database_name
    FROM DISK = '\\UNC_path\full_backup_directory'
[;]

Restore a full user database backup and then a differential backup.
RESTORE DATABASE database_name
    FROM DISK = '\\UNC_path\differential_backup_directory'
    WITH [ ( ] BASE = '\\UNC_path\full_backup_directory' [ ) ]
[;]

Restore header information for a full or differential user database backup.
RESTORE HEADERONLY
    FROM DISK = '\\UNC_path\backup_directory'
[;]

Arguments

RESTORE DATABASE database_name

Specifies to restore a user database to a database called database_name. The restored database can have a different name than the source database that was backed up. database_name cannot already exist as a database on the destination appliance. For more details on permitted database names, see Object Naming Rules (SQL Server PDW).

Restoring a user database restores a full database backup and then optionally restores a differential backup to the appliance. A restore of a user database includes restoring database users, and database roles.

FROM DISK = '\\UNC_path\backup_directory'

The network path and directory from which SQL Server PDW will restore the backup files. For example, FROM DISK = '\\xxx.xxx.xxx.xxx\backups\2012\Monthly\08.2012.Mybackup'.

backup_directory

Specifies the name of a directory that contains the full or differential backup. For example, you can perform a RESTORE HEADERONLY operation on a full or differential backup.

full_backup_directory

Specifies the name of a directory that contains the full backup.

differential_backup_directory

Specifies the name of the directory that contains the differential backup.

  • The path and backup directory must already exist and must be specified as a fully qualified universal naming convention (UNC) path.

  • The path to the backup directory cannot be a local path and it cannot be a location on any of the SQL Server PDW appliance nodes.

  • The maximum length of the UNC path and backup directory name is 200 characters.

  • The server or host must be specified as an IP address.

RESTORE HEADERONLY

Specifies to return only the header information for one user database backup. Among other fields, the header includes the text description of the backup, and the backup name. The backup name does not need to be the same as the name of the directory that stores the backup files.

RESTORE HEADERONLY results are patterned after the SQL Server RESTORE HEADERONLY results. The result has over 50 columns, which are not all used by SQL Server PDW. For a description of the columns in the SQL Server RESTORE HEADERONLY results, see RESTORE HEADERONLY (Transact-SQL) on MSDN.

Permissions

Requires the CREATE ANY DATABASE permission.

Requires a Windows account that has permission to access and read from the backup directory. You must also store the Windows account name and password in SQL Server PDW.

  1. To verify the credentials are already there, use sys.dm_pdw_network_credentials (SQL Server PDW)

  2. To add or update the credentials, use sp_pdw_add_network_credentials (SQL Server PDW).

  3. To remove credentials from SQL Server PDW, use sp_pdw_remove_network_credentials (SQL Server PDW).

Error Handling

The RESTORE DATABASE command results in errors under the following conditions:

  • The name of the database to restore already exists on the target appliance. To avoid this, choose a unique database name, or drop the existing database before running the restore.

  • There is an invalid set of backup files in the backup directory.

  • The login permissions are not sufficient to restore a database.

  • SQL Server PDW does not have the correct permissions to the network location where the backup files are located.

  • The network location for the backup directory does not exist, or is not available.

  • There is insufficient disk space on the Compute nodes or Control node. SQL Server PDW does not confirm that sufficient disk space exists on the appliance before initiating the restore. Therefore, it is possible to generate an out-of-disk-space error while running the RESTORE DATABASE statement. When insufficient disk space occurs, SQL Server PDW rolls back the restore.

  • The target appliance to which the database is being restored has fewer Compute nodes than the source appliance from which the database was backed up.

  • The database restore is attempted from within a transaction.

General Remarks

SQL Server PDW tracks the success of database restores. Before restoring a differential database backup, SQL Server PDW verifies the full database restore finished successfully.

After a restore, the user database will have database compatibility level 120. This is true for all databases regardless of their original compatibility level.

Restoring to an Appliance With a Larger Number of Compute Nodes

Restoring a backup to an appliance with a larger number of Compute nodes grows the allocated database size in proportion to the number of Compute nodes.

For example, when restoring a 60 GB database from a 2-node appliance (30 GB per node) to a 6-node appliance, SQL Server PDW creates a 180 GB database (6 nodes with 30 GB per node) on the 6-node appliance. SQL Server PDW initially restores the database to 2 nodes to match the source configuration, and then redistributes the data to all 6 nodes.

After the redistribution each Compute node will contain less actual data and more free space than each Compute node on the smaller source appliance. Use the additional space to add more data to the database. If the restored database size is larger than you need, you can use ALTER DATABASE (SQL Server PDW) to shrink the database file sizes.

Limitations and Restrictions

For these limitations and restrictions, the source appliance is the appliance from which the database backup was created, and the target appliance is the appliance to which the database will be restored.

Restoring a database does not automatically rebuild statistics.

Only one RESTORE DATABASE or BACKUP DATABASE statement can be running on the appliance at any given time. If multiple backup and restore statements are submitted concurrently, the appliance will put them into a queue and process them one at a time.

You can only restore a database backup to a SQL Server PDW target appliance that has the same number or more Compute nodes than the source appliance. The target appliance cannot have fewer Compute nodes than the source appliance.

You cannot restore a backup that was created on an appliance that has SQL Server 2012 PDW hardware to an appliance that has SQL Server 2008 R2 hardware. This holds true even if the appliance was originally purchased with the SQL Server 2008 R2 PDW hardware and is now running SQL Server 2012 PDW software.

Locking

Takes an exclusive lock on the DATABASE object.

Examples

A. Simple RESTORE examples

The following example restores a full backup to the SalesInvoices2013 database. The backup files are stored in the \\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Full directory. The SalesInvoices2013 database cannot already exist on the target appliance or this command will fail with an error.

Copy image Copy Code
RESTORE DATABASE SalesInvoices2013
FROM DISK = '\\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Full';

B. Restore a full and differential backup

The following example restores a full, and then a differential backup to the SalesInvoices2013 database

The full backup of the database is restored from the full backup which is stored in the '\\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Full' directory. If the restore completes successfully, the differential backup is restored to the SalesInvoices2013 database. The differential backup is stored in the '\\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Diff' directory.

Copy image Copy Code
RESTORE DATABASE SalesInvoices2013
    FROM DISK = '\\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Diff'
    WITH BASE = '\\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Full'
[;]

C. Restoring the backup header

This example restores the header information for database backup '\\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Full' . The command results in one row of information for the Invoices2013Full backup.

Copy image Copy Code
RESTORE HEADERONLY
    FROM DISK = '\\xxx.xxx.xxx.xxx\backups\yearly\Invoices2013Full'
[;]

You can use the header information to check the contents of a backup, or to make sure the target restoration appliance is compatible with the source backup appliance before attempting to restore the backup.

See Also

Reference

BACKUP DATABASE (SQL Server PDW)


TRUNCATE TABLE (SQL Server PDW)

[image]

TRUNCATE TABLE (SQL Server PDW)
See Also Send Feedback

Removes all rows from a SQL Server PDW table, and does not log the individual row deletions.

Syntax

Visual Basic
TRUNCATE TABLE [ { database_name . [ schema_name ] . | schema_name . ] table_name
[;]

Arguments

[ database_name . [schema_name ] . | schema_name . ] table_name.

The name of the table to truncate. The table name can optionally include the schema, or the database and schema.

General Remarks

The table can be distributed or replicated.

The table can be permanent or temporary.

The table can be rowstore or columnstore.

Both TRUNCATE TABLE and DELETE without the WHERE clause result in removing all rows from the table. TRUNCATE TABLE performs faster than DELETE because it does not log the individual row deletions.

Truncating the table removes all pages in the table. This is true for both distributed and replicated tables.

Batch statements and stored procedures can contain TRUNCATE TABLE.

When TRUNCATE TABLE is cancelled before completion, the table remains intact with no data changes.

When truncating a table, SQL Server PDW keeps and updates all statistics objects, both user-defined and auto-generated. Indexes on the table are also updated. After TRUNCATE TABLE completes, all statistics are updated by using the default row count (1000).

While TRUNCATE TABLE is running, all other operations on the table will be blocked.

Limitations and Restrictions

TRUNCATE TABLE is not allowed within the EXPLAIN statement.

TRUNCATE TABLE is not allowed within a user-defined transaction.

Logging Behavior

Truncating a columnstore table is minimally logged.

Performance

For a replicated table, TRUNCATE TABLE is executed in parallel across the compute nodes.

For a distributed table, TRUNCATE TABLE is executed in parallel across the compute nodes, and serially across the distributions within each Compute node.

Permissions

Requires ALTER permission on the target table, table ownership, membership in one of the following roles:

  • sysadmin fixed server role

  • db_owner fixed database role

  • db_ddladmin fixed database role

Differences from SQL Server

Behavior | SQL Server | SQL Server PDW
Run TRUNCATE TABLE within a transaction. | yes | no

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Definition Statements (SQL Server PDW)
SQL Reference (SQL Server PDW)


UPDATE STATISTICS (SQL Server PDW)

[image]

UPDATE STATISTICS (SQL Server PDW)
See Also Send Feedback

Updates query optimization statistics on a SQL Server PDW table. Use the UPDATE STATISTICS statement to improve query performance. For example, we recommend updating statistics after loads.

For more information and best practices for updating statistics, see Understanding Query Optimization Statistics (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

UPDATE STATISTICS [ database_name . [ schema_name ] . |  schema_name . ] table_name
    [ ( { statistics_name | index_name } ) ]
    [ WITH FULLSCAN ]
[;]

Arguments

[ database_name . [ schema_name ] . | schema_name . ] table_name

The one-, two-, or three-part name of the table that contains the statistics to update.

statistics_name | index_name

The statistics to update. This can be specified as the name of a statistics object, or as the name of an index. Since all indexes have statistics, providing the index name easily indicates which statistics to update.

If omitted, SQL Server PDW will update all statistics on the table.

WITH FULLSCAN

Update statistics by scanning all rows in the table. When omitted, SQL Server PDW uses sampling to create the statistics, and determines the sample size that is required to create a high quality query plan.

For table data stored in SQL Server PDW (e.g., replicated tables and distributed tables), SQL Server determines the sample size.

Permissions

Requires ALTER permission on the table, or the user must be the owner of the table, or the user must be a member of the db_ddladmin fixed database role.

Error Handling

If UPDATE STATISTICS returns an error, re-run the command to ensure that statistics are updated on all Compute nodes.

General Remarks

For a distributed table, updating statistics encompasses updating SQL Server statistics on the Compute nodes, and then updating the corresponding merged MPP statistics on the Control node. For more information, see Understanding Query Optimization Statistics (SQL Server PDW).

Limitations and Restrictions

Updating statistics on external tables is not supported. To update external table statistics, drop the statistics with DROP STATISTICS (SQL Server PDW)and then re-create the statistics with CREATE STATISTICS (SQL Server PDW).

Examples

A. Update statistics on a table

The following example updates the CustomerStats1 statistics on the Customer table.

Copy image Copy Code
UPDATE STATISTICS Customer ( CustomerStats1 );

B. Update statistics by using a full scan

The following example updates the CustomerStats1 statistics, based on scanning all of the rows in the Customer table.

Copy image Copy Code
UPDATE STATISTICS Customer (CustomerStats1) WITH FULLSCAN;

C. Update all statistics on a table

The following example updates all statistics on the Customer table.

Copy image Copy Code
UPDATE STATISTICS Customer;

See Also

Reference

CREATE STATISTICS (SQL Server PDW)
DROP STATISTICS (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Query (SQL Server PDW)


Data Manipulation Statements (SQL Server PDW)

[image]

Data Manipulation Statements (SQL Server PDW)
See Also Send Feedback

SQL Server PDW includes data definition statements for manipulation of tables, indexes, logins, and other database objects.

Data manipulation statements are shown in the following table.

Command | Description
DELETE | Removes rows from a table.
INSERT | Adds rows to a table.
UPDATE | Changes existing data in a table.

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Definition Statements (SQL Server PDW)
Queries (SQL Server PDW)


DELETE (SQL Server PDW)

[image]

DELETE (SQL Server PDW)
See Also Send Feedback

Removes rows from a table in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DELETE FROM [database_name . [ schema ] . | schema. ] table_name
    [ WHERE <search_condition> ]
    [ OPTION ( <query_options> [ ,...n ]  ) ]
[; ]

Arguments

FROM [database_name . [schema ] . | schema. ] table_name

The name of the table that will have rows removed. Unlike SQL Server, only one table is allowed. The <from_clause> described in the FROM (SQL Server PDW) topic does not apply to the DELETE statement.

WHERE <search_condition>

Specifies the rows to be deleted. When the WHERE clause is omitted, all rows will be removed from the table. You cannot use a join to delete rows from multiple tables. You can use a join in a subquery of the search condition. For more details about the search conditions, see Search Condition (SQL Server PDW).

OPTION ( <query_options> [ ,...n ] )

Specifies one or more query options, including a label and query join hints. OPTION query hints apply to the entire DELETE statement. For more details, see OPTION (SQL Server PDW).

Error Handling

When a DELETE statement encounters an arithmetic error (overflow, divide by zero, or a domain error) during expression evaluation, the rest of the batch is canceled and an error message is returned.

General Remarks

To delete all the rows in a table, use the DELETE statement without a WHERE clause.

A join hint is allowed, but ignored, if it is specified in a DELETE statement that does not have a join. For example, the following statement runs, but the join hint is ignored.

Copy image Copy Code
DELETE FROM
    AdventureWorksPDW2012
  .dbo.DatabaseLog
WHERE 1=2
OPTION (HASH JOIN)
;

Permissions

Requires DELETE permission on the target table, or membership in the db_datawriter fixed database role. SELECT permissions are also required if the statement contains a WHERE clause. DELETE permission is denied to members of the db_denydatawriter fixed database role.

Limitations and Restrictions

SET ROWCOUNT (SQL Server PDW) has no effect on this statement. To achieve a similar behavior, use TOP (SQL Server PDW).

Locking

Takes an exclusive lock at some level on the table being changed.

Examples

A. Delete all rows from a table

The following example deletes all rows from the Table1 table because a WHERE clause is not used to limit the number of rows deleted.

Copy image Copy Code
DELETE FROM Table1;

B. DELETE a set of rows from a table

The following example deletes all rows from the Table1 table that have a value greater than 1000.00 in the StandardCost column.

Copy image Copy Code
DELETE FROM Table1
WHERE StandardCost > 1000.00;

C. Using LABEL with a DELETE statement

The following example uses a label with the DELETE statement.

Copy image Copy Code
DELETE FROM Table1
OPTION ( LABEL = N'label1' );

D. Using a label and a query hint with the DELETE statement

This query shows the basic syntax for using a query join hint with the DELETE statement. For more information on join hints and how to use the OPTION clause, see OPTION (SQL Server PDW).

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
DELETE FROM dbo.FactInternetSales
WHERE ProductKey IN (
    SELECT T1.ProductKey FROM dbo.DimProduct T1
    JOIN dbo.DimProductSubcategory T2
    ON T1.ProductSubcategoryKey = T2.ProductSubcategoryKey
    WHERE T2.EnglishProductSubcategoryName = 'Road Bikes' )
OPTION ( LABEL = N'CustomJoin', HASH JOIN ) ;

See Also


INSERT (SQL Server PDW)

[image]

INSERT (SQL Server PDW)
See Also Send Feedback

Appends one or more new rows to a table in SQL Server PDW. You can add one row by listing the row values or you can add multiple rows by inserting the results of a SELECT statement.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

INSERT INTO [ database_name . [ schema_name ] . | schema_name . ] table_name
    [ ( column_name [ ,...n ] ) ]
    {
      VALUES ( { NULL | expression } [ ,...n ] )
      | SELECT <select_criteria>
    }
    [ OPTION ( <query_option> [ ,...n ] ) ]
[;]

Arguments

database_name

The name of the database that is to receive the data. The default is the current database.

schema_name

The schema of the table or view.

table_name

The name of the table or view that is to receive the data.

column_name [ ,...n ]

A list of one or more columns in table_name. The list of columns corresponds to the list of values being inserted. The columns do not need to be listed in the same order as the table definition. If the list of columns is omitted, the list of values needs to contain all the columns in the order they are defined in the table definition.

If the list of columns is present and a column is not in the list, SQL Server PDW must be able to insert a value based on the definition of the column; otherwise, the row cannot be loaded. For example, if a column is not listed in the column list and the column is nullable, SQL Server PDW automatically inserts NULL into the column.

VALUES ( { NULL | expression } [ ,...n ] )

The list of one or more data values to insert as a new row in the table. If a list of columns is specified, each column in the column list must contain one value and the values must be in the same order as the columns listed. If the column list is omitted, each column in the table must contain one value and the values must be in the same order as the column definition for the table.

expression

A constant or an expression. At this time, functions are not supported in expression.

SELECT <select_criteria>

A SELECT statement specifying source data that matches the column types to be inserted into the table. For more information, see SELECT (SQL Server PDW).

OPTION ( <query_option> [ ,...n ] )

Specifies query options for the INSERT statement. The possible options include a query label and one or more query join hints. For more information about the OPTION clause, see OPTION (SQL Server PDW)

Permissions

Requires INSERT permission on the target table or membership in the db_datawriter fixed database role. INSERT permission is denied to members of the db_denydatawriter fixed database role.

Error Handling

If an INSERT statement has a value incompatible with the data type of the column, the statement fails and an error message is returned.

If INSERT is loading multiple rows with SELECT, any violation that occurs from the values being loaded (such as an invalid value) stops the statement and no rows are loaded.

General Remarks

Joins are supported with INSERT-SELECT, and are not supported for INSERT with values.

When an INSERT-SELECT is based on a self-join, a table joined to itself, the INSERT operations use the order of columns listed in the FROM clause.

For information about supported data formats, implicit data conversions, and explicit data conversions, see Load Data With INSERT (SQL Server PDW).

For distributed tables, SQL Server PDW performs inserts in parallel across the Compute nodes, and sequentially across the distributions within each Compute node. This provides the ability to rollback inserts in case an insert operation fails.

Limitations and Restrictions

The INSERT statement cannot be used to modify or replace existing data; to modify column values in existing rows, use the Update statement.Examples

SET ROWCOUNT (SQL Server PDW) has no effect on this statement. To achieve a similar behavior, use TOP (SQL Server PDW).

Locking

INSERT (into replicated tables) requires an exclusive lock. INSERT into distributed table acquire SharedUpdate locks at some level on the table being changed.

Examples

A. Using a simple INSERT statement

The following example inserts one row in the Production.UnitMeasure table. Because values for all columns are supplied and are listed in the same order as the columns in the table, the column names do not have to be specified in the column list.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
GO
INSERT INTO DimCurrency
VALUES (500, N'C1', N'Currency1');

B. Inserting data that is not in same order as table columns

The following example uses a column list to explicitly specify the values inserted into each column. The column order in the DimCurrency table is CurrencyKey, CurrencyAlternateKey, CurrencyName. However, the columns are not listed in that order in column_list.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
GO
INSERT INTO DimCurrency (CurrencyName, CurrencyKey, CurrencyAlternateKey)
VALUES (N'Currency1', 500, N'C1');

C. Inserting data using the SELECT option

The following example shows how to insert multiple rows of data using an INSERT statement with a SELECT option. The first INSERT statement uses a SELECT statement directly to retrieve data from the source table, and then to store the result set in the EmployeeTitles table.

Copy image Copy Code
CREATE TABLE EmployeeTitles
( EmployeeKey   INT NOT NULL,
  LastName     varchar(40) NOT NULL,
  Title      varchar(50) NOT NULL
);
INSERT INTO EmployeeTitles
    SELECT EmployeeKey, LastName, Title
    FROM
    AdventureWorksPDW2012
  .dbo.DimEmployee
    WHERE EndDate IS NULL;

D. Specifying a label with the INSERT statement

The following example shows the use of a label with an INSERT statement.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

INSERT INTO DimCurrency
VALUES (500, N'C1', N'Currency1')
OPTION ( LABEL = N'label1' );

E. Using a label and a query hint with the INSERT statement

This query shows the basic syntax for using a label and a query join hint with the INSERT statement. After the query is submitted to the Control node, SQL Server, running on the Compute nodes, will apply the hash join strategy when it generates the SQL Server query plan. For more information on join hints and how to use the OPTION clause, see OPTION (SQL Server PDW).

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
INSERT INTO DimCustomer (CustomerKey, CustomerAlternateKey, FirstName, MiddleName, LastName )
SELECT ProspectiveBuyerKey, ProspectAlternateKey, FirstName, MiddleName, LastName
FROM ProspectiveBuyer p JOIN DimGeography g ON p.PostalCode = g.PostalCode
WHERE g.CountryRegionCode = 'FR'
OPTION ( LABEL = 'Add French Prospects', HASH JOIN)
;

See Also

Reference

UPDATE (SQL Server PDW)
DELETE (SQL Server PDW)


UPDATE (SQL Server PDW)

[image]

UPDATE (SQL Server PDW)
See Also Send Feedback

Changes existing data in a SQL Server PDW table.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

UPDATE [ database_name . [ schema_name ] . | schema_name . ] table_name
SET { column_name = { expression | NULL } } [ ,...n ]
[ FROM from_clause ]
[ WHERE <search_condition> ]
[ OPTION ( LABEL = label_name ) ]
[;]

Arguments

database_name

The name of the database that is to receive the data. The default is the current database.

schema_name

The schema of the table or view.

table_name

The name of the table to update.

from_clause

Specifies a source table or view for the update. For more information, see FROM (SQL Server PDW).

column_name

A column that contains the data to be changed. column_name must exist in table_name.

expression

A literal value, expression, or a subselect statement (enclosed within parentheses) that returns a single value. The value returned by expression replaces the existing value in column_name.

search_condition

Specifies the condition to be met for the rows to be updated. The search condition cannot contain a join. There is no limit to the number of predicates that can be included in a search condition. For more information about predicates and search conditions, see Search Condition (SQL Server PDW).

label_name

A user-provided label name for the query, where label_name is a character string constant enclosed in single quotes. Labels identify queries in the query cache. For more information, see Constants (SQL Server PDW). The string literal can be a Unicode or non-Unicode string literal. For Unicode string literals, use of N prefix is allowed: (N’unicode_string’).

General Remarks

If an update to a row violates a constraint for the column, or if the new value is an incompatible data type, the statement is canceled, an error is returned, and no records are updated.

The UPDATE statement cannot be used to modify the Distribution Key on a distributed table. See CREATE TABLE (SQL Server PDW) for information on Distribution Keys.

When an UPDATE statement encounters an arithmetic error (overflow, divide by zero, or a domain error) during expression evaluation, the update is not performed. The rest of the batch is not run, and an error message is returned.

For distributed tables, SQL Server PDW performs updates in parallel across the Compute nodes, and sequentially across the distributions within each Compute node. This provides the ability to rollback updates in case an update operation fails.

All char and nchar columns are right-padded to the defined length.

The TOP clause cannot be used with the UPDATE statement. For more information, see TOP (SQL Server PDW)

Limitations and Restrictions

UPDATE statements cannot contain the TOP clause. For more information, see TOP (SQL Server PDW).

SET ROWCOUNT (SQL Server PDW) has no effect on this statement.

UPDATE statements cannot contain a FROM clause and a subquery. For example, the following statement is not allowed:

Copy image Copy Code
UPDATE A SET a2=2 FROM B WHERE EXISTS ( SELECT 1 FROM C );

UPDATE statements cannot contain explicit joins, or more than one table, in the FROM clause. For example, the following statement is not allowed:

Copy image Copy Code
UPDATE dbo.Table2
SET dbo.Table2.ColB = dbo.Table2.ColB + dbo.Table1.ColB
FROM dbo.Table2
INNER JOIN dbo.Table1
ON ( dbo.Table2.ColA = dbo.Table1.ColA );

The above query can be rewritten as an implicit join, and with only one table in the FROM clause. The following statement is allowed:

Copy image Copy Code
UPDATE dbo.Table2
SET dbo.Table2.ColB = dbo.Table2.ColB + dbo.Table1.ColB
FROM dbo.Table1
WHERE dbo.Table2.ColA = dbo.Table1.ColA;

Permissions

Requires UPDATE permissions on the target table or membership in the db_datawriter fixed database role. SELECT permissions are also required for the table being updated if the UPDATE statement contains a WHERE clause, or if expression in the SET clause uses a column in the table. UPDATE permission is denied to members of the db_denydatawriter fixed database role.

Locking

Takes an exclusive lock at some level on the table being changed.

Examples

A. Using a simple UPDATE statement

The following examples show how all rows can be affected when a WHERE clause is not used to specify the row (or rows) to update.

This example updates the values in the EndDate and CurrentFlag columns for all rows in the DimEmployee table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

UPDATE DimEmployee
SET EndDate = '2010-12-31', CurrentFlag='False';

You can also use computed values in an UPDATE statement. The following example doubles the value in the ListPrice column for all rows in the Product table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

UPDATE DimEmployee
SET BaseRate = BaseRate * 2;

B. Using the UPDATE statement with a WHERE clause

The following example uses the WHERE clause to specify which rows to update.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

UPDATE DimEmployee
SET FirstName = 'Gail'
WHERE EmployeeKey = 500;

C. Using the UPDATE statement with label

The following example shows use of a LABEL for the UPDATE statement.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

UPDATE DimProduct
SET ProductSubcategoryKey = 2
WHERE ProductKey = 313
OPTION (LABEL = N'label1');

D. Using the UPDATE statement with information from another table

This example creates a table to store total sales by year. It updates the total sales for the year 2004 by running a SELECT statement against the FactInternetSales table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

CREATE TABLE YearlyTotalSales (
    YearlySalesAmount money NOT NULL,
    Year smallint NOT NULL )
WITH ( DISTRIBUTION = REPLICATE );

INSERT INTO YearlyTotalSales VALUES (0, 2004);
INSERT INTO YearlyTotalSales VALUES (0, 2005);
INSERT INTO YearlyTotalSales VALUES (0, 2006);

UPDATE YearlyTotalSales
SET YearlySalesAmount=
(SELECT SUM(SalesAmount) FROM FactInternetSales WHERE OrderDateKey >=20040000 AND OrderDateKey < 20050000)
WHERE Year=2004;

SELECT * FROM YearlyTotalSales;

See Also

Reference

SELECT (SQL Server PDW)
INSERT (SQL Server PDW)
DELETE (SQL Server PDW)


Database Console Commands (SQL Server PDW)

[image]

Database Console Commands (SQL Server PDW)
See Also Send Feedback

The SQL programming language provides DBCC statements that act as Database Console Commands for SQL Server PDW.

At this time, SQL Server PDW supports the following Database Console Command statements:

  1. DBCC FREEPROCCACHE (SQL Server PDW)

  2. DBCC SHRINKLOG (SQL Server PDW)

  3. DBCC PDW_SHOWEXECUTIONPLAN (SQL Server PDW)

  4. DBCC PDW_SHOWPARTITIONSTATS (SQL Server PDW)

  5. DBCC PDW_SHOWSPACEUSED (SQL Server PDW)

See Also


DBCC FREEPROCCACHE (SQL Server PDW)

[image]

DBCC FREEPROCCACHE (SQL Server PDW)
See Also Send Feedback

Removes all query plan cache entries for the current database context from the Compute nodes. Use this statement to force SQL Server PDW to recompile all queries for the current database context the next time they are run.

For example, when an existing query plan was generated with out-of-date statistics, you can improve query performance by updating statistics and then recompiling the query. To force SQL Server PDW to recompile the query instead of using the existing query plan, you can use DBCC FREEPROCCACHE to remove the existing query plans. Upon the next execution of the query, SQL Server PDW will recompile the query with the latest statistics.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DBCC FREEPROCCACHE ( COMPUTE )
     [ WITH NO_INFOMSGS ]
[;]

Arguments

COMPUTE

Purge the query plan cache from each Compute node in the SQL Server PDW appliance. This is the only option available in this release.

WITH NO_INFOMSGS

Results do not contain informational messages.

Permissions

Requires ALTER SERVER STATE permission.

General Remarks

Multiple DBCC FREEPROCCACHE commands can be run concurrently.

In SQL Server PDW, clearing the query plan cache can cause a temporary decrease in query performance when queries are recompiled. DBCC FREEPROCCACHE only causes SQL Server to recompile queries when they are run on the Compute nodes. It does not cause SQL Server PDW to recompile the parallel query plan that is generated on the Control node.

DBCC FREEPROCCACHE can be cancelled during execution.

Error Handling

Errors are reported in the sys.dm_pdw_errors (SQL Server PDW) system view.

Limitations and Restrictions

DBCC FREEPROCCACHE can run within a transaction, however, it cannot be rolled back if the transaction is rolled back.

Metadata

A new row is added to the sys.pdw_exec_requests system view when DBCC FREEPROCCACHE is run.

Examples

A. DBCC FREEPROCCACHE Basic Syntax Examples

The following example removes existing query plan caches from the Compute nodes. The context is set to UserDbSales and so the query plan caches that pertain to that database will be removed. The WITH NO_INFOMSGS clause prevents informational messages from appearing in the results.

Copy image Copy Code
USE UserDbSales;
DBCC FREEPROCCACHE (COMPUTE) WITH NO_INFOMSGS;

The following example has the same results as the previous example, except that informational messages will show in the results.

Copy image Copy Code
USE UserDbSales;
DBCC FREEPROCCACHE (COMPUTE);

When informational messages are requested and the execution is successful, the query results will have one line per Compute node.

B. Granting Permission to run DBCC FREEPROCCACHE

The following example gives the login David permission to run DBCC FREEPROCCACHE.

Copy image Copy Code
GRANT ALTER SERVER STATE TO David;

C. DBCC FREEPROCCACHE Cannot Be Rolled Back

DBCC FREEPROCCACHE can run within a transaction, however, it cannot be rolled back if the transaction is rolled back. Therefore, the following sequence of statements is equivalent to running DBCC FREEPROCCACHE without the transaction context.

Copy image Copy Code
START TRANSACTION;
DBCC FREEPROCCACHE (COMPUTE)
ROLLBACK;

The following example is a transaction that includes an INSERT statement and a DBCC FREEPROCCACHE statement. If the INSERT statement succeeds and DBCC FREEPROCCACHE statement fails, the transaction will be cancelled and the INSERT statement will be rolled back.

Copy image Copy Code
START TRANSACTION;
INSERT INTO T VALUES (a);
DBCC FREEPROCCACHE (COMPUTE)
ROLLBACK;

See Also


DBCC SHRINKLOG (SQL Server PDW)

[image]

DBCC SHRINKLOG (SQL Server PDW)
See Also Send Feedback

Reduces the size of the transaction log across the appliance for the current SQL Server PDW database. The data is defragmented in order to shrink the transaction log. Over time, the database transaction log can become fragmented and inefficient. Use DBCC SHRINKLOG to reduce fragmentation and reduce the log size.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DBCC SHRINKLOG
    [ ( SIZE = { target_size [ MB | GB | TB ]  } | DEFAULT ) ]
    [ WITH NO_INFOMSGS ]
[;]

Arguments

SIZE = { target_size [ MB | GB | TB ] } | DEFAULT.

target_size is the desired size for the transaction log, across all the Compute nodes, after DBCC SHRINKLOG completes. It is an integer greater than 0.

The log size is measured in megabytes (MB), gigabytes (GB), or terabytes (TB). It is the combined size of the transaction log on all of the Compute nodes.

By default, DBCC SHRINKLOG reduces the transaction log to the log size stored in the metadata for the database. The log size in the metadata is determined by the LOG_SIZE parameter in CREATE DATABASE (SQL Server PDW) or ALTER DATABASE (SQL Server PDW). DBCC SHRINKLOG reduces the transaction log size to the default size when SIZE=DEFAULT is specified, or when the SIZE clause is omitted.

WITH NO_INFOMSGS

Informational messages are not displayed in the DBCC SHRINKLOG results.

Permissions

Requires ALTER SERVER STATE permission.

General Remarks

DBCC SHRINKLOG does not change the log size stored in the metadata for the database. The metadata continues to contain the LOG_SIZE parameter that was specified in CREATE DATABASE or ALTER DATABASE statement.

Examples

A. Shrink the transaction log to the original size specified by CREATE DATABASE.

Suppose the transaction log for the Addresses database was set to 100 MB when the Addresses database was created. That is, the CREATE DATABASE statement for Addresses had LOG_SIZE = 100 MB. Now, suppose the log has grown to 150 MB and you want to shrink it back to 100 MB.

Each of the following statements will attempt to shrink the transaction log for the Addresses database to the default size of 100 MB. If shrinking the log to 100 MB will cause data loss, DBCC SHRINKLOG will shrink the log to the smallest size possible, greater than 100 MB, without losing data.

Copy image Copy Code
USE Addresses;
DBCC SHRINKLOG ( SIZE = 100 MB );
DBCC SHRINKLOG ( SIZE = DEFAULT );
DBCC SHRINKLOG;

See Also


DBCC PDW_SHOWEXECUTIONPLAN (SQL Server PDW)

[image]

DBCC PDW_SHOWEXECUTIONPLAN (SQL Server PDW)
See Also Send Feedback

Displays the SQL Server execution plan for a query running on a specific SQL Server PDW Compute node or Control node. SQL Server PDW. Use this to troubleshoot query performance problems while queries are running on the Compute nodes and Control node.

Once query performance problems are understood for SMP SQL Server queries running on the Compute nodes, there are several ways to improve performance. Possible ways to improve query performance on the Compute nodes include creating multi-column statistics, creating non-clustered indexes, or using query hints.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DBCC PDW_SHOWEXECUTIONPLAN ( pdw_node_id, spid )
[;]

Arguments

pdw_node_id

Identifier for the node that is running the query plan. This is an integer and cannot be NULL.

spid

Identifier for the SQL Server session that is running the query plan. This is an integer and cannot be NULL.

Permissions

Requires VIEW-SERVER-STATE permission.

Examples

A. DBCC PDW_SHOWEXECUTIONPLAN Basic Syntax Examples

The query that is running too long is either running a DMS query plan operation or a SQL query plan operation. For a list of the operations and their types, see Understanding Query Plans (SQL Server PDW).

If the query is running a DMS query plan operation, you can use the following query to retrieve a list of the node IDs and session IDs for steps that are not complete.

Copy image Copy Code
SELECT [sql_spid], [pdw_node_id], [request_id], [dms_step_index], [type], [start_time], [end_time], [status]
FROM sys.dm_pdw_dms_workers
WHERE [status] <> 'StepComplete' and [status] <> 'StepError'
AND pdw_node_id = 201001
order by request_id, [dms_step_index], [distribution_id];

Based on the results of the preceding query, use the sql_spid and pdw_node_id as parameters to DBCC PDW_SHOWEXEUCTIONPLAN. For example, the following command shows the execution plan for pdw_node_id 201001 and sql_spid 375.

Copy image Copy Code
DBCC PDW_SHOWEXECUTIONPLAN ( 201001, 375 );

See Also


DBCC PDW_SHOWPARTITIONSTATS (SQL Server PDW)

[image]

DBCC PDW_SHOWPARTITIONSTATS (SQL Server PDW)
See Also Send Feedback

Displays the size and number of rows for each partition of a table in a SQL Server PDW database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

Show the partition stats for a table
DBCC PDW_SHOWPARTITIONSTATS ( " [ database_name . [ schema_name ] . ] | [ schema_name.] table_name  ")
[;]

Arguments

[ database_name . [ schema_name ] . | schema_name . ] table_name

The one, two, or three-part name of the table to be displayed. For two or three-part table names, the name must be enclosed with double quotes (""). Using quotes around a one-part table name is optional.

Permissions

Requires VIEW SERVER STATE permission.

Result Sets

This is the results for the DBCC PDW_SHOWPARTITIONSTATS command.

Column Name | Data Type | Description
partition_number | int | Partition number.
used_page_count | bigint | Number of pages used for the data.
reserved_page_count | bigint | Number of pages allocated to the partition.
row_count | bigint | Number of rows in the partition.
pdw_node_id | int | Compute node for the data.
distribution_id | int | Distribution id for the data.

Examples

A. DBCC PDW_SHOWPARTITIONSTATS Basic Syntax Examples

The following examples display the space used and number of rows by partition for the FactInternetSales table in the AdventureWorksPDW2012 database.

Copy image Copy Code
DBCC PDW_SHOWPARTITIONSTATS ("
    AdventureWorksPDW2012
  .dbo.FactInternetSales");
DBCC PDW_SHOWPARTITIONSTATS ("dbo.FactInternetSales");
DBCC PDW_SHOWPARTITIONSTATS (FactInternetSales);

See Also


DBCC PDW_SHOWSPACEUSED (SQL Server PDW)

[image]

DBCC PDW_SHOWSPACEUSED (SQL Server PDW)
See Also Send Feedback

Displays the number of rows, disk space reserved, and disk space used for a specific table, or for all tables in a SQL Server PDW database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

Show the space used for all user tables
and system tables in the current database
DBCC PDW_SHOWSPACEUSED
[;]

Show the space used for a table
DBCC PDW_SHOWSPACEUSED ( " [ database_name . [ schema_name ] . ] | [ schema_name .] table_name  " )
[;]

Arguments

[ database_name . [ schema_name ] . | schema_name . ] table_name

The one, two, or three-part name of the table to be displayed. For two or three-part table names, the name must be enclosed with double quotes (""). Using quotes around a one-part table name is optional. When no table name is specified, the information is displayed for the current database.

Permissions

Requires VIEW SERVER STATE permission.

Result Sets

This is the result set for all tables.

Column | Data Type | Description
reserved_space | bigint | Total space used for the database, in KB.
data_space | bigint | Space used for data, in KB.
index_space | bigint | Space used for indexes, in KB.
unused_space | bigint | Space that is part of the reserved space and not used, in KB.
pdw_node_id | int | Compute node that is being used for the data.

This is the result set for one table.

Column | Data Type | Description | Range
rows | bigint | Number of rows. |
reserved_space | bigint | Total space reserved for the object, in KB. |
data_space | bigint | Space used for the data, in KB. |
index_space | bigint | Space used for indexes, in KB. |
unused_space | bigint | Space that is part of the reserved space and not used, in KB. |
pdw_node_id | int | Compute node that is used for reporting the space usage. |
distribution_id | int | Distribution that is used for reporting the space usage. | Value is -1 for replicated tables.

Examples

A. DBCC PDW_SHOWSPACEUSED Basic Syntax Examples

The following examples show multiple ways to display the number of rows, disk space reserved, and disk space used by the FactInternetSales table in the AdventureWorksPDW2012 database.

Copy image Copy Code
USE AdventureWorksPDW2012;
DBCC PDW_SHOWSPACEUSED ( "AdventureWorksPDW2012.dbo.FactInternetSales" );
DBCC PDW_SHOWSPACEUSED ( "AdventureWorksPDW2012..FactInternetSales" );
DBCC PDW_SHOWSPACEUSED ( "dbo.FactInternetSales" );
DBCC PDW_SHOWSPACEUSED ( FactInternetSales );

B. Show the disk space used by all tables in the current database

The following example shows the disk space reserved and used by all user tables and system tables in the AdventureWorksPDW2012 database.

Copy image Copy Code
USE AdventureWorksPDW2012;
DBCC PDW_SHOWSPACEUSED;

See Also


DBCC SHOW_STATISTICS (PDW)

[image]

DBCC SHOW_STATISTICS (PDW)
See Also Send Feedback

Displays the contents of query optimization statistics tables stored in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DBCC SHOW_STATISTICS ( table_name , target )
    [ WITH {STAT_HEADER | DENSITY_VECTOR | HISTOGRAM } [ ,...n ] ]
[;]

Arguments

table_name

Name of the table that contains the statistics to display. The table cannot be an external table.

target

Name of the index or statistics object for which to display statistics. Target is enclosed in brackets, single quotes, double quotes, or no quotes.

Different from SQL Server, target cannot be a column name.

STAT_HEADER | DENSITY_VECTOR | HISTOGRAM

Limits the result sets returned by the statement to the specified option or options. If no options are specified, all statistics information is returned.

Results

Each of the three options has a result set.

STAT_HEADER

The following table describes the columns returned in the result set when STAT_HEADER is specified.

Column name | Description
Name | Name of the statistics object.
Updated | Date and time the statistics were last updated. The STATS_DATE function is an alternate way to retrieve this information.
Rows | Total number of rows in the table or indexed view when the statistics were last updated.
Rows Sampled | Total number of rows sampled for statistics calculations. If Rows Sampled < Rows, the displayed histogram and density results are estimates based on the sampled rows.
Steps | Number of steps in the histogram. Each step spans a range of column values followed by an upper bound column value. The histogram steps are defined on the first key column in the statistics. The maximum number of steps is 200.
Density | Calculated as 1 / distinct values for all values in the first key column of the statistics object, excluding the histogram boundary values. This Density value is not used by the query optimizer and is displayed for backward compatibility with versions before SQL Server 2008.
Average Key Length | Average number of bytes per value for all of the key columns in the statistics object.
String Index | Yes indicates the statistics object contains string summary statistics to improve the cardinality estimates for query predicates that use the LIKE operator; for example, WHERE ProductName LIKE '%Bike'. String summary statistics are stored separately from the histogram and are created on the first key column of the statistics object when it is of type char, varchar, nchar, nvarchar, varchar(max), nvarchar(max), text, or ntext..
Filter Expression | Predicate for the subset of table rows included in the statistics object. NULL = non-filtered statistics.
Unfiltered Rows | Total number of rows in the table before applying the filter expression. If Filter Expression is NULL, Unfiltered Rows is equal to Rows.

DENSITY_VECTOR

The following table describes the columns returned in the result set when DENSITY_VECTOR is specified.

Column name | Description
All Density | Density is 1 / distinct values. Results display density for each prefix of columns in the statistics object, one row per density. A distinct value is a distinct list of the column values per row and per columns prefix. For example, if the statistics object contains key columns (A, B, C), the results report the density of the distinct lists of values in each of these column prefixes: (A), (A,B), and (A, B, C). Using the prefix (A, B, C), each of these lists is a distinct value list: (3, 5, 6), (4, 4, 6), (4, 5, 6), (4, 5, 7). Using the prefix (A, B) the same column values have these distinct value lists: (3, 5), (4, 4), and (4, 5)
Average Length | Average length, in bytes, to store a list of the column values for the column prefix. For example, if the values in the list (3, 5, 6) each require 4 bytes the length is 12 bytes.
Columns | Names of columns in the prefix for which All density and Average length are displayed.

HISTOGRAM

The following table describes the columns returned in the result set when the HISTOGRAM option is specified.

Column name | Description
RANGE_HI_KEY | Upper bound column value for a histogram step. The column value is also called a key value.
RANGE_ROWS | Estimated number of rows whose column value falls within a histogram step, excluding the upper bound.
EQ_ROWS | Estimated number of rows whose column value equals the upper bound of the histogram step.
DISTINCT_RANGE_ROWS | Estimated number of rows with a distinct column value within a histogram step, excluding the upper bound.
AVG_RANGE_ROWS | Average number of rows with duplicate column values within a histogram step, excluding the upper bound (RANGE_ROWS / DISTINCT_RANGE_ROWS for DISTINCT_RANGE_ROWS > 0).

General Remarks

Histogram

A histogram measures the frequency of occurrence for each distinct value in a data set. The query optimizer computes a histogram on the column values in the first key column of the statistics object, selecting the column values by statistically sampling the rows or by performing a full scan of all rows in the table or view. If the histogram is created from a sampled set of rows, the stored totals for number of rows and number of distinct values are estimates and do not need to be whole integers.

To create the histogram, the query optimizer sorts the column values, computes the number of values that match each distinct column value and then aggregates the column values into a maximum of 200 contiguous histogram steps. Each step includes a range of column values followed by an upper bound column value. The range includes all possible column values between boundary values, excluding the boundary values themselves. The lowest of the sorted column values is the upper boundary value for the first histogram step.

The following diagram shows a histogram with six steps. The area to the left of the first upper boundary value is the first step.

For each histogram step:

  • Bold line represents the upper boundary value (RANGE_HI_KEY) and the number of times it occurs (EQ_ROWS)

  • Solid area left of RANGE_HI_KEY represents the range of column values and the average number of times each column value occurs (AVG_RANGE_ROWS). The AVG_RANGE_ROWS for the first histogram step is always 0.

  • Dotted lines represent the sampled values used to estimate total number of distinct values in the range (DISTINCT_RANGE_ROWS) and total number of values in the range (RANGE_ROWS). The query optimizer uses RANGE_ROWS and DISTINCT_RANGE_ROWS to compute AVG_RANGE_ROWS and does not store the sampled values.

The query optimizer defines the histogram steps according to their statistical significance. It uses a maximum difference algorithm to minimize the number of steps in the histogram while maximizing the difference between the boundary values. The maximum number of steps is 200. The number of histogram steps can be fewer than the number of distinct values, even for columns with fewer than 200 boundary points. For example, a column with 100 distinct values can have a histogram with fewer than 100 boundary points.

Density Vector

The query optimizer uses densities to enhance cardinality estimates for queries that return multiple columns from the same table or indexed view. The density vector contains one density for each prefix of columns in the statistics object. For example, if a statistics object has the key columns CustomerId, ItemId, Price, density is calculated on each of the following column prefixes.

Column prefix | Density calculated on
(CustomerId) | Rows with matching values for CustomerId
(CustomerId, ItemId) | Rows with matching values for CustomerId and ItemId
(CustomerId, ItemId, Price) | Rows with matching values for CustomerId, ItemId, and Price

Permissions

DBCC SHOW_STATISTICS requires SELECT permission on the table or membership in one of the following:

  • sysadmin fixed server role

  • db_owner fixed database role

  • db_ddladmin fixed database role

Limitations and Restrictions

DBCC SHOW_STATISTICS shows statistics stored in the Shell database at the Control node level. It does not show statistics that are auto-created by SQL Server on the Compute nodes. For more information, see Understanding Query Optimization Statistics (SQL Server PDW).

DBCC SHOW_STATISTICS is not supported on external tables.

Examples

A. Display the contents of one statistics object

The following example displays the contents of the Customer_LastName statistics on the DimCustomer table.

Copy image Copy Code
--First, create a statistics object
USE AdventureWorksPDW2012;
GO
CREATE STATISTICS Customer_LastName
ON AdventureWorksPDW2012.dbo.DimCustomer (LastName);
GO
DBCC SHOW_STATISTICS ("dbo.DimCustomer",Customer_LastName);
GO

The results show the header, the density vector, and part of the histogram.

DBCC SHOW_STATISTICS results

B. Specifying the HISTOGRAM option

This limits the statistics information displayed for Customer_LastName to the HISTOGRAM data.

Copy image Copy Code
DBCC SHOW_STATISTICS ("dbo.DimCustomer",Customer_LastName) WITH HISTOGRAM;
GO

See Also

Reference

CREATE STATISTICS (SQL Server PDW)

Concepts

Understanding Query Optimization Statistics (SQL Server PDW)


Queries (SQL Server PDW)

[image]

Queries (SQL Server PDW)
See Also Send Feedback

SQL Server PDW includes data definition statements for manipulation of tables, indexes, logins, and other database objects.

This topic is organized by object type, as shown:

SELECT

SELECT Clauses

Supporting Topics

SELECT

Command | Description
SELECT | Retrieves rows from a table or view.

SELECT Clauses

Command | Description
WITH | Begins a common table expression.
TOP | Returns only a specified number of rows.
FROM | Specifies source tables or views.
GROUP BY | Groups a selected set of rows by the value of one more columns or expressions.
HAVING | Specifies a search condition applied to a group or aggregate.
ORDER BY | Specifies the sort order for returned values.
OVER | Determines the partitioning and order of a rowset before an analytic function is applied.
UNION | Combines the rules of two or more queries into a single result set.
EXCEPT | Returns any distinct values from the left query that are not also found on the right query.
INTERSECT | Returns any distinct values that are returned by both the query on the left and right sides of the INTERSECT operand.
WHERE | Specifies the search condition for the rows returned by the query.

Supporting Topics

Statement | Description
Aliasing | Allows temporary substitution of a short string in place of a table or column name.
EXPLAIN | Returns the query plan for a statement.
Search Condition | A combination of one or more predicates that use the logical operators AND, OR, and NOT.
Subqueries | A query that is nested inside another statement or subquery.

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Definition Statements (SQL Server PDW)
Data Manipulation Statements (SQL Server PDW)


SELECT (SQL Server PDW)

[image]

SELECT (SQL Server PDW)
See Also Send Feedback

SQL query statement that retrieves data from tables or views in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

[ WITH <common_table_expression> [ ,...n ] ]
SELECT <select_criteria>
[;]

<select_criteria> ::=
    [ TOP ( top_expression ) ]
    [ ALL | DISTINCT ]
    { * | column_name | expression } [ ,...n ]
    [ FROM { table_source } [ ,...n ] ]
    [ WHERE <search_condition> ]
    [ GROUP BY <group_by_clause> ]
    [ HAVING <search_condition> ]
    [ ORDER BY <order_by_expression> ]
    [ OPTION ( <query_option> [ ,...n ] ) ]

Arguments

common_table_expression

Specifies a temporary named result set, known as a common table expression (CTE). For more information, see WITH common_table_expression (SQL Server PDW).

TOP ( top_expression )

Returns only a specified top set of rows from the query result set. top_expression is a number.

For more information, see TOP (SQL Server PDW).

ALL | DISTINCT

All specifies that duplicate rows can appear in the results. DISTINCT specifies that only unique rows will be returned in the results.

{ * | column_name | expression } [ ,... n]

The list of one or more columns or expressions to be selected for the result set.

*

Returns all columns from all tables and views in the FROM clause; the columns are returned in the order in which they exist in the table or view.

column_name

column_name is an expression. The maximum number of expressions that can be specified in the select list is 4096. For information on specifying a different name for a column, see Aliasing (SQL Server PDW).

expression

expression can be a variable or a function that is used without a FROM clause. For example, SELECT @@VERSION, SELECT CURRENT_TIMESTAMP.

FROM { <table_source> } [ ,...n ]

Limits the scope of the result set to a set of tables or views. For more information, see FROM (SQL Server PDW).

WHERE < search_condition >

Limits the scope of the returned values based on conditions. For more information, see WHERE (SQL Server PDW).

GROUP BY < group_by_clause >

Defines a grouping for the result set. For more information, see GROUP BY (SQL Server PDW).

HAVING < search_condition >

Specifies requirements on the group or aggregate. For more information, see HAVING (SQL Server PDW).

ORDER BY < order_by_expression >

Orders the result set. For more information, see ORDER BY (SQL Server PDW).

OPTION ( <query_option> [ ,...n ] )

Specifies a query option, which is either a query label or a query join hint. A query label is useful for locating a query within the Admin Console. A query hint might be useful to improve query performance when SQL Server, running on the Compute nodes, does not generate an optimal query plan. For more information, see OPTION (SQL Server PDW).

Permissions

Requires SELECT permission on the table or view.

General Remarks

Queries on columnstore tables run in memory as much as possible to achieve maximum performance. SQL Server PDW has pre-set resource governor settings that moderate the memory available for in-memory queries. If a query does not have enough in-memory resources, SQL Server PDW uses disk resources to complete the query without failing.

Use the UNION operator between queries to combine their results into one result set. For more information, see UNION (SQL Server PDW).

The order of the clauses in the SELECT statement is significant. Any one of the optional clauses can be omitted, but optional clauses must appear in the appropriate order when they are used.

SQL Server PDW raises an exception and rolls back the current running statement if a SELECT statement produces a result row or an intermediate work table row that exceeds 8,060 bytes.

Processing Order of WHERE, GROUP BY, HAVING Clauses

The following steps show the processing order for a SELECT statement with a WHERE clause, a GROUP BY clause, and a HAVING clause:

  1. The FROM clause returns an initial result set.

  2. The WHERE clause excludes rows not meeting its search condition.

  3. The GROUP BY clause collects the selected rows into one group for each unique value in the GROUP BY clause.

  4. Aggregate functions specified in the SELECT list calculate summary values for each group.

  5. The HAVING clause additionally excludes rows not meeting its search condition.

Limitations and Restrictions

SELECT statements cannot contain distinct aggregates on constant expressions. For example, the following is not supported: SELECT COUNT ( DISTINCT 3 ) FROM Table A.

Locking

Takes a shared lock at some level of detail on the table being queried.

Examples

A. Using SELECT to retrieve rows and columns

This section shows three code examples. This first code example returns all rows (no WHERE clause is specified) and all columns (using the *) from the DimEmployee table in the AdventureWorksPDW2012 database.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT *
FROM DimEmployee
ORDER BY LastName;

This next example using table aliasing to achieve the same result.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT e.*
FROM DimEmployee AS e
ORDER BY LastName;

This example returns all rows (no WHERE clause is specified) and a subset of the columns (FirstName, LastName, StartDate) from the DimEmployee table in the AdventureWorksPDW2012 database. The third column heading is renamed to FirstDay.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, StartDate AS FirstDay
FROM DimEmployee
ORDER BY LastName;

This example returns only the rows for DimEmployee that have an EndDate that is not NULL and a MaritalStatus of ‘M’ (married).

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, StartDate AS FirstDay
FROM DimEmployee
WHERE EndDate IS NOT NULL
AND MaritalStatus = 'M'
ORDER BY LastName;

B. Using SELECT with column headings and calculations

The following example returns all rows from the DimEmployee table, and calculates the gross pay for each employee based on their BaseRate and a 40 hour work week.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT FirstName, LastName, BaseRate, BaseRate * 40 AS GrossPay
FROM DimEmployee
ORDER BY LastName;

C. Using DISTINCT with SELECT

The following example uses DISTINCT to generate a list of all unique titles in the DimEmployee table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT DISTINCT Title
FROM DimEmployee
ORDER BY Title;

D. Using GROUP BY

The following example finds the total amount for all sales on each day.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT OrderDateKey, SUM(SalesAmount) AS TotalSales
FROM FactInternetSales
GROUP BY OrderDateKey
ORDER BY OrderDateKey;

Because of the GROUP BY clause, only one row containing the sum of all sales is returned for each day.

E. Using GROUP BY with multiple groups

The following example finds the average price and the sum of Internet sales for each day, grouped by order date and the promotion key.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT OrderDateKey, PromotionKey, AVG(SalesAmount) AS AvgSales, SUM(SalesAmount) AS TotalSales
FROM FactInternetSales
GROUP BY OrderDateKey, PromotionKey
ORDER BY OrderDateKey;

F. Using GROUP BY and WHERE

The following example puts the results into groups after retrieving only the rows with order dates later than August 1, 2002.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT OrderDateKey, SUM(SalesAmount) AS TotalSales
FROM FactInternetSales
WHERE OrderDateKey > '20020801'
GROUP BY OrderDateKey
ORDER BY OrderDateKey;

G. Using GROUP BY with an expression

The following example groups by an expression. You can group by an expression if the expression does not include aggregate functions.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT SUM(SalesAmount) AS TotalSales
FROM FactInternetSales
GROUP BY (OrderDateKey * 10);

H. Using GROUP BY with ORDER BY

The following example finds the sum of sales per day, and orders by the day.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT OrderDateKey, SUM(SalesAmount) AS TotalSales
FROM FactInternetSales
GROUP BY OrderDateKey
ORDER BY OrderDateKey;

I. Using the HAVING clause

This query uses the HAVING clause to restrict results.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT OrderDateKey, SUM(SalesAmount) AS TotalSales
FROM FactInternetSales
GROUP BY OrderDateKey
HAVING OrderDateKey > 20010000
ORDER BY OrderDateKey;

See Also


WITH common_table_expression (SQL Server PDW)

[image]

WITH common_table_expression (SQL Server PDW)
See Also Send Feedback

Specifies a temporary named result set, known as a common table expression (CTE). This is derived from a simple query and defined within the execution scope of a single SELECT statement. This clause can also be used in a CREATE VIEW, CREATE TABLE AS SELECT, CREATE EXTERNAL TABLE AS SELECT, or CREATE REMOTE TABLE AS SELECT statement as part of its defining SELECT statement.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

[ WITH <common_table_expression> [ ,...n ] ]

<common_table_expression>::=
    expression_name [ ( column_name [ ,...n ] ) ]
    AS
    ( CTE_query_definition )

Arguments

expression_name

Is a valid identifier for the common table expression. expression_name must be different from the name of any other common table expression defined in the same WITH <common_table_expression> clause, but expression_name can be the same as the name of a base table or view. Any reference to expression_name in the query uses the common table expression and not the base object.

column_name

Specifies a column name in the common table expression. Duplicate names within a single CTE definition are not allowed. The number of column names specified must match the number of columns in the result set of the CTE_query_definition. The list of column names is optional only if distinct names for all resulting columns are supplied in the query definition.

CTE_query_definition

Specifies a SELECT statement whose result set populates the common table expression. The SELECT statement for CTE_query_definition must meet the same requirements as for creating a view, except a CTE cannot define another CTE.

If more than one CTE_query_definition is defined, the query definitions must be joined by one of these set operators: UNION ALL, UNION, or EXCEPT.

Remarks

Features and Limitations of Common Table Expressions

The current implementation of CTEs in SQL Server PDW has the following features and limitations.

  • A CTE can be specified in a SELECT statement.

  • A CTE can be specified in a CREATE VIEW statement.

  • A CTE can be specified in a CREATE TABLE AS SELECT (CTAS) statement.

  • A CTE can be specified in a CREATE REMOTE TABLE AS SELECT (CRTAS) statement.

  • A CTE can be specified in a CREATE EXTERNAL TABLE AS SELECT (CETAS) statement.

  • A remote table can be referenced from a CTE.

  • An external table can be referenced from a CTE.

  • Multiple CTE query definitions can be defined in a CTE.

  • A CTE must be followed by a single SELECT statement. INSERT, UPDATE, DELETE, and MERGE statements are not supported.

  • A common table expression that includes references to itself (a recursive common table expression) is not supported.

  • Specifying more than one WITH clause in a CTE is not allowed. For example, if a CTE_query_definition contains a subquery, that subquery cannot contain a nested WITH clause that defines another CTE.

  • An ORDER BY clause cannot be used in the CTE_query_definition, except when a TOP clause is specified.

  • When a CTE is used in a statement that is part of a batch, the statement before it must be followed by a semicolon.

  • When used in statements prepared by sp_prepare, CTEs will behave the same way as other SELECT statements in PDW. However, if CTEs are used as part of CETAS prepared by sp_prepare, the behavior can defer from SQL Server and other PDW statements because of the way binding is implemented for sp_prepare. If SELECT that references CTE is using a wrong column that does not exist in CTE, the sp_prepare will pass without detecting the error, but the error will be thrown during sp_execute instead.

For standard Guidelines for Creating and Using Common Table Expressions and Guidelines for Defining and Using Recursive Common Table Expressions, see those sections of SQL Server Books Online at WITH common_table_expression (Transact-SQL).

Examples

A. Creating a simple common table expression

The following example shows the total number of sales orders per year for each sales representative at Adventure Works Cycles.

Copy image Copy Code
USE AdventureWorks2012;
GO
-- Define the CTE expression name and column list.
WITH Sales_CTE (SalesPersonID, SalesOrderID, SalesYear)
AS
-- Define the CTE query.
(
    SELECT SalesPersonID, SalesOrderID, YEAR(OrderDate) AS SalesYear
    FROM Sales.SalesOrderHeader
    WHERE SalesPersonID IS NOT NULL
)
-- Define the outer query referencing the CTE name.
SELECT SalesPersonID, COUNT(SalesOrderID) AS TotalSales, SalesYear
FROM Sales_CTE
GROUP BY SalesYear, SalesPersonID
ORDER BY SalesPersonID, SalesYear;
GO

B. Using a common table expression to limit counts and report averages

The following example shows the average number of sales orders for all years for the sales representatives.

Copy image Copy Code
WITH Sales_CTE (SalesPersonID, NumberOfOrders)
AS
(
    SELECT SalesPersonID, COUNT(*)
    FROM Sales.SalesOrderHeader
    WHERE SalesPersonID IS NOT NULL
    GROUP BY SalesPersonID
)
SELECT AVG(NumberOfOrders) AS "Average Sales Per Person"
FROM Sales_CTE;
GO

C. Using a common table expression within a CTAS statement

The following example creates a new table containing the total number of sales orders per year for each sales representative at Adventure Works Cycles.

Copy image Copy Code
USE AdventureWorks2012;
GO

CREATE TABLE SalesOrdersPerYear
WITH
(
    DISTRIBUTION = HASH(SalesPersonID)
)
AS
    -- Define the CTE expression name and column list.
    WITH Sales_CTE (SalesPersonID, SalesOrderID, SalesYear)
    AS
    -- Define the CTE query.
    (
        SELECT SalesPersonID, SalesOrderID, YEAR(OrderDate) AS SalesYear
        FROM Sales.SalesOrderHeader
        WHERE SalesPersonID IS NOT NULL
    )
    -- Define the outer query referencing the CTE name.
    SELECT SalesPersonID, COUNT(SalesOrderID) AS TotalSales, SalesYear
    FROM Sales_CTE
    GROUP BY SalesYear, SalesPersonID
    ORDER BY SalesPersonID, SalesYear;
GO

D. Using a common table expression within a CETAS statement

The following example creates a new external table containing the total number of sales orders per year for each sales representative at Adventure Works Cycles.

Copy image Copy Code
USE AdventureWorks2012;
GO

CREATE EXTERNAL TABLE SalesOrdersPerYear
WITH
(
    LOCATION = 'hdfs://xxx.xxx.xxx.xxx:5000/files/Customer',
    FORMAT_OPTIONS ( FIELD_TERMINATOR = '|' )
)
AS
    -- Define the CTE expression name and column list.
    WITH Sales_CTE (SalesPersonID, SalesOrderID, SalesYear)
    AS
    -- Define the CTE query.
    (
        SELECT SalesPersonID, SalesOrderID, YEAR(OrderDate) AS SalesYear
        FROM Sales.SalesOrderHeader
        WHERE SalesPersonID IS NOT NULL
    )
    -- Define the outer query referencing the CTE name.
    SELECT SalesPersonID, COUNT(SalesOrderID) AS TotalSales, SalesYear
    FROM Sales_CTE
    GROUP BY SalesYear, SalesPersonID
    ORDER BY SalesPersonID, SalesYear;
GO

E. Using multiple comma separated CTEs in a statement

The following example demonstrates including two CTEs in a single statement. The CTEs cannot be nested (no recursion).

Copy image Copy Code
WITH
 CountDate (TotalCount, TableName) AS
    (
     SELECT COUNT(datekey), 'DimDate' FROM DimDate
    ) ,
 CountCustomer (TotalAvg, TableName) AS
    (
     SELECT COUNT(CustomerKey), 'DimCustomer' FROM DimCustomer
    )
SELECT TableName, TotalCount FROM CountDate
UNION ALL
SELECT TableName, TotalAvg FROM CountCustomer;

See Also

Reference

CREATE VIEW (SQL Server PDW)
UNION (SQL Server PDW)
SELECT (SQL Server PDW)
CREATE TABLE AS SELECT (SQL Server PDW)
CREATE REMOTE TABLE AS SELECT (SQL Server PDW)
CREATE EXTERNAL TABLE AS SELECT (SQL Server PDW)


EXCEPT and INTERSECT (SQL Server PDW)

[image]

EXCEPT and INTERSECT (SQL Server PDW)
See Also Send Feedback

Returns distinct rows by comparing the results of two queries.

EXCEPT returns distinct rows from the left input query that aren’t output by the right input query.

INTERSECT returns distinct rows that are output by both the left and right input queries.

The basic rules for combining the result sets of two queries that use EXCEPT or INTERSECT are the following:

  • The number and the order of the columns must be the same in all queries.

  • The data types must be compatible.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

{ <query_specification> | ( <query_expression> ) }
{ EXCEPT | INTERSECT }
{ <query_specification> | ( <query_expression> ) }

Arguments

<query_specification> | ( <query_expression> )

Is a query specification or query expression that returns data to be compared with the data from another query specification or query expression. The definitions of the columns that are part of an EXCEPT or INTERSECT operation do not have to be the same, but they must be comparable through implicit conversion. When data types differ, the type that is used to perform the comparison and return results is determined based on the rules for data type precedence.

When the types are the same but differ in precision, scale, or length, the result is determined based on the same rules for combining expressions. For more information, see Precision, Scale, and Length (Transact-SQL).

The query specification or expression cannot return xml, text, ntext, image, or nonbinary CLR user-defined type columns because these data types are not comparable.

EXCEPT

EXCEPT returns distinct rows from the left input query that aren’t output by the right input query.

INTERSECT

Returns any distinct values that are returned by both the query on the left and right sides of the INTERSECT operator.

Remarks

When the data types of comparable columns that are returned by the queries to the left and right of the EXCEPT or INTERSECT operators are character data types with different collations, the required comparison is performed according to the rules of collation precedence. If this conversion cannot be performed, the Database Engine returns an error.

When comparing column values for determining DISTINCT rows, two NULL values are considered equal.

The column names of the result set that are returned by EXCEPT or INTERSECT are the same names as those returned by the query on the left side of the operator.

Column names or aliases in ORDER BY clauses must reference column names returned by the left-side query.

The nullability of any column in the result set returned by EXCEPT or INTERSECT is the same as the nullability of the corresponding column that is returned by the query on the left side of the operator.

If EXCEPT or INTERSECT is used together with other operators in an expression, it is evaluated in the context of the following precedence:

  1. Expressions in parentheses

  2. The INTERSECT operator

  3. EXCEPT and UNION evaluated from left to right based on their position in the expression

If EXCEPT or INTERSECT is used to compare more than two sets of queries, data type conversion is determined by comparing two queries at a time, and following the previously mentioned rules of expression evaluation.

Examples

The following examples show how to use the INTERSECT and EXCEPT operators. The first query returns all values from the FactInternetSales table for comparison to the results with INTERSECT and EXCEPT.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
SELECT CustomerKey
FROM FactInternetSales;
--Result: 60398 Rows

The following query returns any distinct values that are returned by both the query on the left and right sides of the INTERSECT operator.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
SELECT CustomerKey
FROM FactInternetSales
INTERSECT
SELECT CustomerKey
FROM DimCustomer
WHERE DimCustomer.Gender = 'F'
ORDER BY CustomerKey;
--Result: 9133 Rows (Sales to customers that are female.)

The following query returns any distinct values from the query to the left of the EXCEPT operator that are not also found on the right query.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
SELECT CustomerKey
FROM FactInternetSales
EXCEPT
SELECT CustomerKey
FROM DimCustomer
WHERE DimCustomer.Gender = 'F'
ORDER BY CustomerKey;
--Result: 9351 Rows (Sales to customers that are not female.)

See Also

Reference

UNION (SQL Server PDW)


EXPLAIN (SQL Server PDW)

[image]

EXPLAIN (SQL Server PDW)
See Also Send Feedback

Returns the query plan for a SQL Server PDW SQL statement without running the statement. Use EXPLAIN to preview which operations will require data movement and to view the estimated costs of the query operations.

For more information about query plans, see Understanding Query Plans (SQL Server PDW).

Syntax

EXPLAIN SQL_statement
[;]

Arguments

SQL_statement

The SQL statement on which EXPLAIN will run. SQL_statement can be any of these commands: SELECT, INSERT, UPDATE, DELETE, CREATE TABLE AS SELECT, CREATE REMOTE TABLE.

Permissions

Requires the SHOWPLAN permission, and permission to execute SQL_statement. See Permissions: GRANT, DENY, REVOKE (SQL Server PDW).

Return Value

The return value from the EXPLAIN command is an XML document with the structure shown below. This XML document lists all operations in the query plan for the given query, each enclosed by the <dsql_operation> tag. The return value is of type nvarchar(max).

The returned query plan depicts sequential SQL statements; when the query runs it may involve parallelized operations, so some of the sequential statements shown may run at the same time.

Copy image Copy Code
<?xml version="1.0" encoding="utf-8"?>
<dsql_query>
  <sql>. . .</sql>
  <params />
  <dsql_operations>
    <dsql_operation>
     . . .
    </dsql_operation>
    [ . . . n ]
  <dsql_operations>
</dsql_query>

The XML tags contain this information:

XML Tag

Summary, Attributes, and Content

<dsql_query>

Top level/document element.

<sql>

Echoes SQL_statement.

<params>

This tag is not used at this time.

<dsql_operations>

Summarizes and contains the query steps, and includes cost information for the query. Also contains all of the <dsql_operation> blocks. This tag contains count information for the entire query:

<dsql_operations total_cost=total_cost total_number_operations=total_number_operations>

  • total_cost is the total estimated time for the query to run, in ms.

  • total_number_operations is the total number of operations for the query. An operation that will be parallelized and run on multiple nodes is counted as a single operation.

<dsql_operation>

Describes a single operation within the query plan. The <dsql_operation> tag contains the operation type as an attribute:

<dsql_operation operation_type=operation_type>

operation_type is one of the values found in Understanding Query Plans (SQL Server PDW).

The content in the <dsql_operation> block is dependent on the operation type.

Operation Type

Content

Example

BROADCAST_MOVE, DISTRIBUTE_REPLICATED_TABLE_MOVE, MASTER_TABLE_MOVE, PARTITION_MOVE, SHUFFLE_MOVE, and TRIM_MOVE

<operation_cost> element, with these attributes. Values reflect only the local operation:

  • cost is the local operator cost and shows the estimated time for the operation to run, in ms.

  • accumulative_cost is the sum of all seen operations in the plan including summed values for parallel operations, in ms.

  • average_rowsize is the estimated average row size (in bytes) of rows retrieved and passed during the operation.

  • output_rows is the output (node) cardinality and shows the number of output rows.

<location>: The nodes or distributions where the operation will occur. Options are: “Control”, “ComputeNode”, “AllComputeNodes”, “AllDistributions”, “SubsetDistributions”, “Distribution”, and “SubsetNodes”.

<source_statement>: The source data for the shuffle move.

<destination_table>: The internal temporary table the data will be moved into.

<shuffle_columns>: (Applicable only to SHUFFLE_MOVE operations). One or more columns that will be used as the distribution columns for the temporary table.

<operation_cost cost="40" accumulative_cost="40" average_rowsize = "50" output_rows="100"/>

<location distribution="AllDistributions" />

<source_statement type="statement">SELECT [TableAlias_3b77ee1d8ccf4a94ba644118b355db9d].[dist_date] FROM [qatest].[dbo].[flyers] [TableAlias_3b77ee1d8ccf4a94ba644118b355db9d] </source_statement>

<destination_table>Q_[TEMP_ID_259]_[PARTITION_ID]</destination_table>

<shuffle_columns>dist_date;</shuffle_columns>

CopyOperation

<operation_cost>: See <operation_cost> above.

<DestinationCatalog>: The destination node or nodes.

<DestinationSchema>: The destination schema in DestinationCatalog.

<DestinationTableName>: Name of the destination table or “TableName”.

<DestinationDatasource>: The name or connection information for the destination datasource.

<Username> and <Password>: These fields indicate that a username and password for the destination may be required.

<BatchSize>: The batch size for the copy operation.

<SelectStatement>: The select statement used to perform the copy.

<distribution>: The distribution where the copy is performed.

<operation_cost cost="0" accumulative_cost="0" average_rowsize="4" output_rows="1" />

<DestinationCatalog>master</DestinationCatalog>

<DestinationSchema>dbo</DestinationSchema>

<DestinationTableName>[TableName]</DestinationTableName>

<DestinationDatasource>localhost, 8080</DestinationDatasource>

<Username>...</Username>

<Password>...</Password>

<BatchSize>6000</BatchSize>

<SelectStatement>SELECT T1_1.c1 AS c1 FROM [qatest].[dbo].[gigs] AS T1_1</SelectStatement>

<distribution>ControlNode</distribution>

MetaDataCreate_Operation

<source_table>: The source table for the operation.

<destionation_table>: The destination table for the operation.

<source_table>databases</source_table>

<destination_table>MetaDataCreateLandingTempTable</destination_table>

ON

<location>: See <location> above.

<sql_operation>: Identifies the SQL command that will be performed on a node.

<location permanent="false" distribution="AllDistributions">Compute</location>

<sql_operation type="statement">CREATE TABLE [tempdb].[dbo]. [Q_[TEMP_ID_259]]_ [PARTITION_ID]]]([dist_date] DATE) WITH (DISTRIBUTION = HASH([dist_date]),) </sql_operation>

RemoteOnOperation

<DestinationCatalog>: The destination catalog.

<DestinationSchema>: The destination schema in DestinationCatalog.

<DestinationTableName>: Name of the destination table or “TableName”.

<DestinationDatasource>: Name of the destination datasource.

<Username> and <Password>: These fields indicate that a username and password for the destination may be required.

<CreateStatement>: The table creation statement for the destination database.

<DestinationCatalog>master</DestinationCatalog>

<DestinationSchema>dbo</DestinationSchema>

<DestinationTableName>TableName</DestinationTableName>

<DestinationDatasource>DestDataSource</DestinationDatasource>

<Username>...</Username>

<Password>...</Password>

<CreateStatement>CREATE TABLE [master].[dbo].[TableName] ([col1] BIGINT) ON [PRIMARY] WITH(DATA_COMPRESSION=PAGE);</CreateStatement>

RETURN

<resultset>: The identifier for the result set.

<resultset>RS_19</resultset>

RND_ID

<identifier>: The identifier for the object created.

<identifier>TEMP_ID_260</identifier>

Limitations and Restrictions

EXPLAIN can be applied to optimizable queries only, which are queries that can be improved or modified based on the results of an EXPLAIN command. The supported EXPLAIN commands are listed above. Attempting to use EXPLAIN with an unsupported query type will either return an error or echo the query.

EXPLAIN is not supported in a user transaction.

Examples

The following example shows an EXPLAIN command run on a SELECT statement, and the XML result.

Submitting an EXPLAIN statement

The submitted command for this example is:

Copy image Copy Code
USE AdventureWorksPDW2012;
EXPLAIN
    SELECT CAST (AVG(YearlyIncome) AS int) AS AverageIncome,
        CAST(AVG(FIS.SalesAmount) AS int) AS AverageSales,
        G.StateProvinceName, T.SalesTerritoryGroup
    FROM dbo.DimGeography AS G
    JOIN dbo.DimSalesTerritory AS T
        ON G.SalesTerritoryKey = T.SalesTerritoryKey
    JOIN dbo.DimCustomer AS C
        ON G.GeographyKey = C.GeographyKey
    JOIN dbo.FactInternetSales AS FIS
        ON C.CustomerKey = FIS.CustomerKey
    WHERE T.SalesTerritoryGroup IN ('North America', 'Pacific')
        AND Gender = 'F'
    GROUP BY G.StateProvinceName, T.SalesTerritoryGroup
    ORDER BY AVG(YearlyIncome) DESC;
GO

After executing the statement using the EXPLAIN option, the message tab presents a single line titled explain, and starting with the XML text <?xml version="1.0" encoding="utf-8"?> Click on the XML to open the entire text in an XML window. To better understand the following comments, you should turn on the display of line numbers in SSDT.

To turn on line numbers

  1. With the output appearing in the explain tab SSDT, on the TOOLS menu, select Options.

  2. Expand the Text Editor section, expand XML, and then click General.

  3. In the Display area, check Line numbers.

  4. Click OK.

Example EXPLAIN output

The XML result of the EXPLAIN command with row numbers turned on is:

Copy image Copy Code
1  <?xml version="1.0" encoding="utf-8"?>
2  <dsql_query>
3    <sql>SELECT CAST (AVG(YearlyIncome) AS int) AS AverageIncome,
4          CAST(AVG(FIS.SalesAmount) AS int) AS AverageSales,
5          G.StateProvinceName, T.SalesTerritoryGroup
6      FROM dbo.DimGeography AS G
7      JOIN dbo.DimSalesTerritory AS T
8          ON G.SalesTerritoryKey = T.SalesTerritoryKey
9      JOIN dbo.DimCustomer AS C
10          ON G.GeographyKey = C.GeographyKey
11      JOIN dbo.FactInternetSales AS FIS
12          ON C.CustomerKey = FIS.CustomerKey
13      WHERE T.SalesTerritoryGroup IN ('North America', 'Pacific')
14          AND Gender = 'F'
15      GROUP BY G.StateProvinceName, T.SalesTerritoryGroup
16      ORDER BY AVG(YearlyIncome) DESC</sql>
17    <dsql_operations total_cost="0.926237696" total_number_operations="9">
18      <dsql_operation operation_type="RND_ID">
19        <identifier>TEMP_ID_16893</identifier>
20      </dsql_operation>
21      <dsql_operation operation_type="ON">
22        <location permanent="false" distribution="AllComputeNodes" />
23        <sql_operations>
24          <sql_operation type="statement">CREATE TABLE [tempdb].[dbo].[TEMP_ID_16893] ([CustomerKey] INT NOT NULL, [GeographyKey] INT, [YearlyIncome] MONEY ) WITH(DATA_COMPRESSION=PAGE);</sql_operation>
25        </sql_operations>
26      </dsql_operation>
27      <dsql_operation operation_type="BROADCAST_MOVE">
28        <operation_cost cost="0.121431552" accumulative_cost="0.121431552" average_rowsize="16" output_rows="31.6228" />
29        <source_statement>SELECT [T1_1].[CustomerKey] AS [CustomerKey],
30         [T1_1].[GeographyKey] AS [GeographyKey],
31         [T1_1].[YearlyIncome] AS [YearlyIncome]
32  FROM   (SELECT [T2_1].[CustomerKey] AS [CustomerKey],
33                 [T2_1].[GeographyKey] AS [GeographyKey],
34                 [T2_1].[YearlyIncome] AS [YearlyIncome]
35          FROM   [AdventureWorksPDW2012].[dbo].[DimCustomer] AS T2_1
36          WHERE  ([T2_1].[Gender] = CAST (N'F' COLLATE Latin1_General_100_CI_AS_KS_WS AS NVARCHAR (1)) COLLATE Latin1_General_100_CI_AS_KS_WS)) AS T1_1</source_statement>
37        <destination_table>[TEMP_ID_16893]</destination_table>
38      </dsql_operation>
39      <dsql_operation operation_type="RND_ID">
40        <identifier>TEMP_ID_16894</identifier>
41      </dsql_operation>
42      <dsql_operation operation_type="ON">
43        <location permanent="false" distribution="AllDistributions" />
44        <sql_operations>
45          <sql_operation type="statement">CREATE TABLE [tempdb].[dbo].[TEMP_ID_16894] ([StateProvinceName] NVARCHAR(50) COLLATE Latin1_General_100_CI_AS_KS_WS, [SalesTerritoryGroup] NVARCHAR(50) COLLATE Latin1_General_100_CI_AS_KS_WS NOT NULL, [col] BIGINT, [col1] MONEY NOT NULL, [col2] BIGINT, [col3] MONEY NOT NULL ) WITH(DATA_COMPRESSION=PAGE);</sql_operation>
46        </sql_operations>
47      </dsql_operation>
48      <dsql_operation operation_type="SHUFFLE_MOVE">
49        <operation_cost cost="0.804806144" accumulative_cost="0.926237696" average_rowsize="232" output_rows="108.406" />
50        <source_statement>SELECT [T1_1].[StateProvinceName] AS [StateProvinceName],
51         [T1_1].[SalesTerritoryGroup] AS [SalesTerritoryGroup],
52         [T1_1].[col2] AS [col],
53         [T1_1].[col] AS [col1],
54         [T1_1].[col3] AS [col2],
55         [T1_1].[col1] AS [col3]
56  FROM   (SELECT ISNULL([T2_1].[col1], CONVERT (MONEY, 0.00, 0)) AS [col],
57                 ISNULL([T2_1].[col3], CONVERT (MONEY, 0.00, 0)) AS [col1],
58                 [T2_1].[StateProvinceName] AS [StateProvinceName],
59                 [T2_1].[SalesTerritoryGroup] AS [SalesTerritoryGroup],
60                 [T2_1].[col] AS [col2],
61                 [T2_1].[col2] AS [col3]
62          FROM   (SELECT   COUNT_BIG([T3_2].[YearlyIncome]) AS [col],
63                           SUM([T3_2].[YearlyIncome]) AS [col1],
64                           COUNT_BIG(CAST ((0) AS INT)) AS [col2],
65                           SUM([T3_2].[SalesAmount]) AS [col3],
66                           [T3_2].[StateProvinceName] AS [StateProvinceName],
67                           [T3_1].[SalesTerritoryGroup] AS [SalesTerritoryGroup]
68                  FROM     (SELECT [T4_1].[SalesTerritoryKey] AS [SalesTerritoryKey],
69                                   [T4_1].[SalesTerritoryGroup] AS [SalesTerritoryGroup]
70                            FROM   [AdventureWorksPDW2012].[dbo].[DimSalesTerritory] AS T4_1
71                            WHERE  (([T4_1].[SalesTerritoryGroup] = CAST (N'North America' COLLATE Latin1_General_100_CI_AS_KS_WS AS NVARCHAR (13)) COLLATE Latin1_General_100_CI_AS_KS_WS)
72                                    OR ([T4_1].[SalesTerritoryGroup] = CAST (N'Pacific' COLLATE Latin1_General_100_CI_AS_KS_WS AS NVARCHAR (7)) COLLATE Latin1_General_100_CI_AS_KS_WS))) AS T3_1
73                           INNER JOIN
74                           (SELECT [T4_1].[SalesTerritoryKey] AS [SalesTerritoryKey],
75                                   [T4_2].[YearlyIncome] AS [YearlyIncome],
76                                   [T4_2].[SalesAmount] AS [SalesAmount],
77                                   [T4_1].[StateProvinceName] AS [StateProvinceName]
78                            FROM   [AdventureWorksPDW2012].[dbo].[DimGeography] AS T4_1
79                                   INNER JOIN
80                                   (SELECT [T5_2].[GeographyKey] AS [GeographyKey],
81                                           [T5_2].[YearlyIncome] AS [YearlyIncome],
82                                           [T5_1].[SalesAmount] AS [SalesAmount]
83                                    FROM   [AdventureWorksPDW2012].[dbo].[FactInternetSales] AS T5_1
84                                           INNER JOIN
85                                           [tempdb].[dbo].[TEMP_ID_16893] AS T5_2
86                                           ON ([T5_1].[CustomerKey] = [T5_2].[CustomerKey])) AS T4_2
87                                   ON ([T4_2].[GeographyKey] = [T4_1].[GeographyKey])) AS T3_2
88                           ON ([T3_1].[SalesTerritoryKey] = [T3_2].[SalesTerritoryKey])
89                  GROUP BY [T3_2].[StateProvinceName], [T3_1].[SalesTerritoryGroup]) AS T2_1) AS T1_1</source_statement>
90        <destination_table>[TEMP_ID_16894]</destination_table>
91        <shuffle_columns>StateProvinceName;</shuffle_columns>
92      </dsql_operation>
93      <dsql_operation operation_type="ON">
94        <location permanent="false" distribution="AllComputeNodes" />
95        <sql_operations>
96          <sql_operation type="statement">DROP TABLE [tempdb].[dbo].[TEMP_ID_16893]</sql_operation>
97        </sql_operations>
98      </dsql_operation>
99      <dsql_operation operation_type="RETURN">
100        <location distribution="AllDistributions" />
101        <select>SELECT   [T1_1].[col] AS [col],
102           [T1_1].[col1] AS [col1],
103           [T1_1].[StateProvinceName] AS [StateProvinceName],
104           [T1_1].[SalesTerritoryGroup] AS [SalesTerritoryGroup],
105           [T1_1].[col2] AS [col2]
106  FROM     (SELECT CONVERT (INT, [T2_1].[col], 0) AS [col],
107                   CONVERT (INT, [T2_1].[col1], 0) AS [col1],
108                   [T2_1].[StateProvinceName] AS [StateProvinceName],
109                   [T2_1].[SalesTerritoryGroup] AS [SalesTerritoryGroup],
110                   [T2_1].[col] AS [col2]
111            FROM   (SELECT CASE
112                            WHEN ([T3_1].[col] = CAST ((0) AS BIGINT)) THEN CAST (NULL AS MONEY)
113                            ELSE ([T3_1].[col1] / CONVERT (MONEY, [T3_1].[col], 0))
114                           END AS [col],
115                           CASE
116                            WHEN ([T3_1].[col2] = CAST ((0) AS BIGINT)) THEN CAST (NULL AS MONEY)
117                            ELSE ([T3_1].[col3] / CONVERT (MONEY, [T3_1].[col2], 0))
118                           END AS [col1],
119                           [T3_1].[StateProvinceName] AS [StateProvinceName],
120                           [T3_1].[SalesTerritoryGroup] AS [SalesTerritoryGroup]
121                    FROM   (SELECT ISNULL([T4_1].[col], CONVERT (BIGINT, 0, 0)) AS [col],
122                                   ISNULL([T4_1].[col1], CONVERT (MONEY, 0.00, 0)) AS [col1],
123                                   ISNULL([T4_1].[col2], CONVERT (BIGINT, 0, 0)) AS [col2],
124                                   ISNULL([T4_1].[col3], CONVERT (MONEY, 0.00, 0)) AS [col3],
125                                   [T4_1].[StateProvinceName] AS [StateProvinceName],
126                                   [T4_1].[SalesTerritoryGroup] AS [SalesTerritoryGroup]
127                            FROM   (SELECT   SUM([T5_1].[col]) AS [col],
128                                             SUM([T5_1].[col1]) AS [col1],
129                                             SUM([T5_1].[col2]) AS [col2],
130                                             SUM([T5_1].[col3]) AS [col3],
131                                             [T5_1].[StateProvinceName] AS [StateProvinceName],
132                                             [T5_1].[SalesTerritoryGroup] AS [SalesTerritoryGroup]
133                                    FROM     [tempdb].[dbo].[TEMP_ID_16894] AS T5_1
134                                    GROUP BY [T5_1].[StateProvinceName], [T5_1].[SalesTerritoryGroup]) AS T4_1) AS T3_1) AS T2_1) AS T1_1
135  ORDER BY [T1_1].[col2] DESC</select>
136      </dsql_operation>
137      <dsql_operation operation_type="ON">
138        <location permanent="false" distribution="AllDistributions" />
139        <sql_operations>
140          <sql_operation type="statement">DROP TABLE [tempdb].[dbo].[TEMP_ID_16894]</sql_operation>
141        </sql_operations>
142      </dsql_operation>
143    </dsql_operations>
144  </dsql_query>

Meaning of the EXPLAIN output

The output above contains 144 numbered lines. Your output from this query may differ slightly. The following list describes significant sections.

  • Lines 3 through 16 provide a description of the query that is being analyzed.

  • Line 17, specifies that the total number of operations will be 9. You can find the start of each operation, by looking for the words dsql_operation.

  • Line 18 starts operation 1. Lines 18 and 19 indicate that a RND_ID operation will create a random ID number that will be used for an object description. The object described in the output above is TEMP_ID_16893. Your number will be different.

  • Line 20 starts operation 2. Lines 21 through 25: On all compute nodes, create a temporary table named TEMP_ID_16893.

  • Line 26 starts operation 3. Lines 27 through 37: Move data to TEMP_ID_16893 by using a broadcast move. The query sent to each compute node is provided. Line 37 specifies the destination table is TEMP_ID_16893.

  • Line 38 starts operation 4. Lines 39 through 40: Create a random ID for a table. TEMP_ID_16894 is the ID number in the example above. Your number will be different.

  • Line 41 starts operation 5. Lines 42 through 46: On all nodes, create a temporary table named TEMP_ID_16894.

  • Line 47 starts operation 6. Lines 48 through 91: Move data from various tables (including TEMP_ID_16893) to table TEMP_ID_16894, by using a shuffle move operation. The query sent to each compute node is provided. Line 90 specifies the destination table as TEMP_ID_16894. Line 91 specifies the columns.

  • Line 92 starts operation 7. Lines 93 through 97: On all compute nodes, drop temporary table TEMP_ID_16893.

  • Line 98 starts operation 8. Lines 99 through 135: Return results to the client. Uses the query provided to get the results.

  • Line 136 starts operation 9. Lines 137 through 140: On all nodes, drop temporary table TEMP_ID_16894.

See Also


FROM (SQL Server PDW)

[image]

FROM (SQL Server PDW)
See Also Send Feedback

Specifies the tables, views, and joined tables that are used in the SQL Server PDW DELETE, SELECT, and UPDATE statements for SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

FROM { <table_source> [ ,...n ] }

<table_source> ::=
{
    [ database_name . [ schema_name ] . | schema_name . ] table_or_view_name [ AS ] table_or_view_alias
    | derived_table [ AS ] table_alias [ ( column_alias [ ,...n ] ) ]
    | <joined_table>
}

<joined_table> ::=
{
    <table_source> <join_type> <table_source> ON search_condition
    | <table_source> CROSS JOIN <table_source>
    | left_table_source { CROSS | OUTER } APPLY right_table_source
    | [ ( ] <joined_table> [ ) ]
}

<join_type> ::=
    [ INNER ] [ <join hint> ] JOIN
    | LEFT  [ OUTER ] JOIN
    | RIGHT [ OUTER ] JOIN
    | FULL  [ OUTER ] JOIN

<join_hint> ::=
    REDUCE
    | REPLICATE
    | REDISTRIBUTE

Arguments

<table_source>

Specifies a table or view (with or without an alias) to use in the SQL statement. Up to 256 table sources can be used in a statement, although the limit varies depending on available memory and the complexity of other expressions in the query. Individual queries may not support up to 256 table sources.

Note Note

Query performance can decrease when many tables are referenced in a query. Query performance is also affected by additional factors, such as the presence of indexes and indexed views on each table source, the size of the select list in the SELECT statement, and whether the query contains complex joins.

table_source can be a one, two, or three part name, specifying the table/view, schema, and database. Fully qualifying the object name can slightly improve performance by reducing the need to resolve the schema, can eliminate errors from assuming a default schema, and can make your code more readable.

The order of table sources after the FROM keyword does not affect the result set that is returned. SQL Server PDW returns errors when duplicate names appear in the FROM clause.

table_or_view_alias

The alias used for the table or view name.

derived _table

A subquery that retrieves rows from the database. derived_table is used as input to the outer query.

<joined_table>

A result set that is the product of two or more tables. For multiple joins, use parentheses to change the natural order of the joins.

<join_type>

Specifies the type of join operation.

[ INNER ] [ <join_hint> ] JOIN

Returns all matching pairs of rows. Discards unmatched rows from both tables. When no join type is specified, this is the default.

FULL [ OUTER ] JOIN

Returns a row from either the left or right table that does not meet the join condition, and sets to NULL the output columns that correspond to the other table. This is in addition to all rows typically returned by inner join.

LEFT [ OUTER ] JOIN

Returns all rows from the left table not meeting the join condition. Sets to NULL the output columns from the other table and all rows returned by the inner join.

RIGHT [OUTER] JOIN

Returns all rows from the right table not meeting the join condition. Sets to NULL the output columns that correspond to the other table and all rows returned by the inner join.

ON search_condition

Specifies the condition on which the join is based. The condition can specify any predicate, although columns and comparison operators are frequently used.

For more information about search conditions and predicates, see Search Condition (SQL Server PDW).

CROSS JOIN

Returns every possible combination of data from all joined tables regardless of relationship. If table1 has 100 rows and table2 has 5 rows, the result set will have 500 rows. Cross-joins are discouraged because the result set can grow unmanageably large.

left_table_source{ CROSS | OUTER } APPLY right_table_source

Specifies that the right_table_source of the APPLY operator is evaluated against every row of the left_table_source. This functionality is useful when the right_table_source contains a table-valued function that takes column values from the left_table_source as one of its arguments.

Either CROSS or OUTER must be specified with APPLY. When CROSS is specified, no rows are produced when the right_table_source is evaluated against a specified row of the left_table_source and returns an empty result set.

When OUTER is specified, one row is produced for each row of the left_table_source even when the right_table_source evaluates against that row and returns an empty result set.

For more information, see the Remarks section.

join_hint

These join hints apply to INNER joins on two distribution incompatible columns. They can improve query performance by restricting the amount of data movement that occurs during query processing.

REDUCE

Reduces the number of rows to be moved for the table on the right side of the join in order to make two distribution incompatible tables compatible. The REDUCE hint is also called a semi-join hint.

REPLICATE

Causes the values in the joining column from the table on the left side of the join to be replicated to all nodes. The table on the right is joined to the replicated version of those columns.

REDISTRIBUTE

Forces two data sources to be distributed on columns specified in the JOIN clause. For a distributed table, SQL Server PDW will perform a shuffle move. For a replicated table, SQL Server PDW will perform a trim move. To understand these move types, see the DMS Query Plan Operations section in Understanding Query Plans (SQL Server PDW). This hint can improve performance when the query plan is using a broadcast move to resolve a distribution incompatible join.

Error Handling

The following error occurs if SQL Server PDW or SQL Server cannot honor the query hint:

Query processor could not produce a query plan because of the hints defined in this query.

General Remarks

The FROM clause is always required in the SELECT statement, even when the select list contains only constants and arithmetic expressions, and does not contain column names.

To join columns of incompatible data types, use an explicit cast to make the columns compatible.

All of the join tables in the ON clause do not need to be referenced in the SELECT list of the SELECT statement.

For outer joins, the location of the join predicates can cause different query results. This is because join predicates in the ON clause of the join are applied before the join, and join predicates in the WHERE clause are applied to the result of the join.

NULL values are not equal when compared in join predicates. For example, an outer join will return each row with a NULL value in the outer table of the join predicate.

REDISTRIBUTE:

The REDISTRIBUTE hint has the following behaviors according to the join predicates in the ON clause.

  1. Distribution compatible join – a join between two distribution columns:

    • REDISTRIBUTE hint is ignored.

  2. Distribution incompatible inner join between a distribution column and a non-distribution column:

    • When there is only one join predicate in the ON clause, the REDISTRIBUTE hint forces a Shuffle move on the non-distribution column.

    • When the ON clause combines multiple join predicates with the AND operator, the REDISTRIBUTE hint forces a Shuffle move on the non-distribution column in the first expression.

  3. Distribution incompatible inner join between two non-distribution columns:

    • When there is only one join predicate in the ON clause, the REDISTRIBUTE hint forces a Shuffle move on both of the non-distribution columns.

    • When the ON clause combines multiple join predicates with the AND operator, the REDISTRIBUTE hint forces a Shuffle move on both of the non-distribution columns in the first predicate.

Limitations and Restrictions

There are limitations for using joins in the UDPATE and DELETE statements. For more information, see UPDATE (SQL Server PDW) and DELETE (SQL Server PDW).

Only one of the join hints (REDUCE, REPLICATE, REDISTRIBUTE) is allowed in the ON clause. A join hint and the DISTRIBUTED_AGG query hint can exist together in the same query.

The INNER keyword must be specified when using a REDUCE, REPLICATE, or REDISTRIBUTE join hint.

For the REDUCE or REDISTRIBUTE hints, the join predicate must be an equijoin; it must only use the equality operator (=) to compare values.

For the REDUCE hint, the ON clause must include one and only one column from each data source in the join. To specify additional join predicates, use the WHERE clause.

Security

Permissions

Requires permissions for the DELETE, SELECT, or UPDATE statement.

Examples

A. Using a simple FROM clause

The following example retrieves the SalesTerritoryID and SalesTerritoryRegion columns from the DimSalesTerritory table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT SalesTerritoryKey, SalesTerritoryRegion
FROM DimSalesTerritory
ORDER BY SalesTerritoryKey;

B. Using the INNER JOIN syntax

The following example returns the SalesOrderNumber, ProductKey, and EnglishProductName columns from the FactInternetSales and DimProduct tables where the join key, ProductKey, matches in both tables. The SalesOrderNumber and EnglishProductName columns each exist in one of the tables only, so it is not necessary to specify the table alias with these columns, as is shown; these aliases are included for readability. The word AS before an alias name is not required but is recommended for readability and to conform to the ANSI standard.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT fis.SalesOrderNumber, dp.ProductKey, dp.EnglishProductName
FROM FactInternetSales AS fis INNER JOIN DimProduct AS dp
ON dp.ProductKey = fis.ProductKey;

Since the INNER keyword is not required for inner joins, this same query could be written as:

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT fis.SalesOrderNumber, dp.ProductKey, dp.EnglishProductName
FROM FactInternetSales fis JOIN DimProduct dp
ON dp.ProductKey = fis.ProductKey;

A WHERE clause could also be used with this query to limit results. This example limits results to SalesOrderNumber values higher than ‘SO5000’:

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT fis.SalesOrderNumber, dp.ProductKey, dp.EnglishProductName
FROM FactInternetSales AS fis JOIN DimProduct AS dp
ON dp.ProductKey = fis.ProductKey
WHERE fis.SalesOrderNumber > 'SO50000'
ORDER BY fis.SalesOrderNumber;

C. Using the LEFT OUTER JOIN and RIGHT OUTER JOIN syntax

The following example joins the FactInternetSales and DimProduct tables on the ProductKey columns. The left outer join syntax preserves the unmatched rows from the left (FactInternetSales) table. Since the FactInternetSales table does not contain any ProductKey values that do not match the DimProduct table, this query returns the same rows as the first inner join example above.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT fis.SalesOrderNumber, dp.ProductKey, dp.EnglishProductName
FROM FactInternetSales AS fis LEFT OUTER JOIN DimProduct AS dp
ON dp.ProductKey = fis.ProductKey;

This query could also be written without the OUTER keyword.

In right outer joins, the unmatched rows from the right table are preserved. The following example returns the same rows as the left outer join example above.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT fis.SalesOrderNumber, dp.ProductKey, dp.EnglishProductName
FROM DimProduct AS dp RIGHT OUTER JOIN FactInternetSales AS fis
ON dp.ProductKey = fis.ProductKey;

The following query uses the DimSalesTerritory table as the left table in a left outer join. It retrieves the SalesOrderNumber values from the FactInternetSales table. If there are no orders for a particular SalesTerritoryKey, the query will return a NULL for the SalesOrderNumber for that row. This query is ordered by the SalesOrderNumber column, so that any NULLs in this column will appear at the top of the results.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT dst.SalesTerritoryKey, dst.SalesTerritoryRegion, fis.SalesOrderNumber
FROM DimSalesTerritory AS dst LEFT OUTER JOIN FactInternetSales AS fis
ON dst.SalesTerritoryKey = fis.SalesTerritoryKey
ORDER BY fis.SalesOrderNumber;

This query could be rewritten with a right outer join to retrieve the same results:

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT dst.SalesTerritoryKey, dst.SalesTerritoryRegion, fis.SalesOrderNumber
FROM FactInternetSales AS fis RIGHT OUTER JOIN DimSalesTerritory AS dst
ON fis.SalesTerritoryKey = dst.SalesTerritoryKey
ORDER BY fis.SalesOrderNumber;

D. Using the FULL OUTER JOIN syntax

The following example demonstrates a full outer join, which returns all rows from both joined tables but returns NULL for values that do not match from the other table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT dst.SalesTerritoryKey, dst.SalesTerritoryRegion, fis.SalesOrderNumber
FROM DimSalesTerritory AS dst FULL OUTER JOIN FactInternetSales AS fis
ON dst.SalesTerritoryKey = fis.SalesTerritoryKey
ORDER BY fis.SalesOrderNumber;

This query could also be written without the OUTER keyword.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT dst.SalesTerritoryKey, dst.SalesTerritoryRegion, fis.SalesOrderNumber
FROM DimSalesTerritory AS dst FULL JOIN FactInternetSales AS fis
ON dst.SalesTerritoryKey = fis.SalesTerritoryKey
ORDER BY fis.SalesOrderNumber;

E. Using the CROSS JOIN syntax

The following example returns the cross-product of the FactInternetSales and DimSalesTerritory tables. A list of all possible combinations of SalesOrderNumber and SalesTerritoryKey are returned. Notice the absence of the ON clause in the cross join query.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT dst.SalesTerritoryKey, fis.SalesOrderNumber
FROM DimSalesTerritory AS dst CROSS JOIN FactInternetSales AS fis
ORDER BY fis.SalesOrderNumber;

F. Using a derived table

The following example uses a derived table (a SELECT statement after the FROM clause) to return the CustomerKey and LastName columns of all customers in the DimCustomer table with BirthDate values later than January 1, 1970 and the last name ‘Smith’.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT CustomerKey, LastName
FROM
   (SELECT * FROM DimCustomer
    WHERE BirthDate > '01/01/1970') AS DimCustomerDerivedTable
WHERE LastName = 'Smith'
ORDER BY LastName;

G. REDUCE join hint example

The following example uses the REDUCE join hint to alter the processing of the derived table within the query. When using the REDUCE join hint in this query, the fis.ProductKey is projected, replicated and made distinct, and then joined to DimProduct during the shuffle of DimProduct on ProductKey. The resulting derived table is distributed on fis.ProductKey.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

EXPLAIN SELECT SalesOrderNumber
FROM
   (SELECT fis.SalesOrderNumber, dp.ProductKey, dp.EnglishProductName
    FROM DimProduct AS dp
      INNER REDUCE JOIN FactInternetSales AS fis
      ON dp.ProductKey = fis.ProductKey
   ) AS dTable
ORDER BY SalesOrderNumber;

H. REPLICATE join hint example

This next example shows the same query as the previous example, except that a REPLICATE join hint is used instead of the REDUCE join hint. Use of the REPLICATE hint causes the values in the ProductKey (joining) column from the FactInternetSales table to be replicated to all nodes. The DimProduct table is joined to the replicated version of those values.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

EXPLAIN SELECT SalesOrderNumber
FROM
   (SELECT fis.SalesOrderNumber, dp.ProductKey, dp.EnglishProductName
    FROM DimProduct AS dp
      INNER REPLICATE JOIN FactInternetSales AS fis
      ON dp.ProductKey = fis.ProductKey
   ) AS dTable
ORDER BY SalesOrderNumber;

I. Using the REDISTRIBUTE hint to guarantee a Shuffle move for a distribution incompatible join

The following query uses the REDISTRIBUTE query hint on a distribution incompatible join. This guarantees the query optimizer will use a Shuffle move in the query plan. This also guarantees the query plan will not use a Broadcast move which moves a distributed table to a replicated table.

In the following example, the REDISTRIBUTE hint forces a Shuffle move on the FactInternetSales table because ProductKey is the distribution column for DimProduct, and is not the distribution column for FactInternetSales.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
EXPLAIN
SELECT dp.ProductKey, fis.SalesOrderNumber, fis.TotalProductCost
FROM DimProduct dp INNER REDISTRIBUTE JOIN FactInternetSales fis
ON dp.ProductKey = fis.ProductKey;

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Queries (SQL Server PDW)


GROUP BY (SQL Server PDW)

[image]

GROUP BY (SQL Server PDW)
See Also Send Feedback

The GROUP BY clause is used within a SELECT statement in SQL Server PDW to group the selected rows into summary rows. One row is returned for each group.

For the SELECT statement, see SELECT (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

GROUP BY <group_by_item> [ ,...n ]

<group_by_item>::=
    column_name [ WITH (DISTRIBUTED_AGG) ]
    | column_expression

Arguments

column_name

The column name on which the grouping operation is performed.

expression

The expression on which the grouping operation is performed. The expression can contain column names, derived tables, and view names. Columns do not need to appear in the select list of the SELECT clause. Views must be referenced in the FROM clause.

WITH (DISTRIBUTED_AGG)

The DISTRIBUTED_AGG query hint forces SQL Server PDW to redistribute a table on a specific column before performing an aggregation. Only one column in the GROUP BY clause can have a DISTRIBUTED_AGG query hint. After the query finishes, the redistributed table is dropped. The original table is not changed.

General Remarks

GROUP BY:

  • If aggregate functions are included in the SELECT clause <select list>, GROUP BY calculates a summary value for each group.

  • Rows that do not meet the conditions of the WHERE clause are removed before any grouping operation is performed.

  • The HAVING (SQL Server PDW) clause is used with the GROUP BY clause to filter groups in the result set.

  • The GROUP BY clause does not order the result set. Use the ORDER BY clause to order the result set.

  • If a grouping column contains null values, all null values are considered equal and they are collected into a single group.

DISTRIBUTED_AGG query hint:

  • The DISTRIBUTED_AGG query hint is provided for backwards compatibility and will not improve performance for most queries. By default, SQL Server PDW already redistributes data as necessary to improve performance for aggregations.

Examples

A. Basic use of the GROUP BY clause

The following example finds the total amount for all sales on each day. One row containing the sum of all sales is returned for each day.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT OrderDateKey, SUM(SalesAmount) AS TotalSales FROM FactInternetSales
GROUP BY OrderDateKey ORDER BY OrderDateKey;

B. Basic use of the DISTRIBUTED_AGG hint

This example uses the DISTRIBUTED_AGG query hint to force the appliance to shuffle the table on the CustomerKey column before performing the aggregation.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
SELECT CustomerKey, SUM(SalesAmount) AS sas
FROM FactInternetSales
GROUP BY CustomerKey WITH (DISTRIBUTED_AGG)
ORDER BY CustomerKey DESC;

C. Syntax Variations for GROUP BY

When the select list has no aggregations, each column in the select list must be included in the GROUP BY list. Computed columns in the select list can be listed, but are not required, in the GROUP BY list. These are examples of syntactically valid SELECT statements:

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
SELECT LastName, FirstName FROM DimCustomer GROUP BY LastName, FirstName;
SELECT NumberCarsOwned FROM DimCustomer GROUP BY YearlyIncome, NumberCarsOwned;
SELECT (SalesAmount + TaxAmt + Freight) AS TotalCost FROM FactInternetSales GROUP BY SalesAmount, TaxAmt, Freight;
SELECT SalesAmount, SalesAmount*1.10 SalesTax FROM FactInternetSales GROUP BY SalesAmount;
SELECT SalesAmount FROM FactInternetSales GROUP BY SalesAmount, SalesAmount*1.10;

D. Using a GROUP BY with multiple GROUP BY expressions

The following example groups results using multiple GROUP BY criteria. If, within each OrderDateKey group, there are subgroups that can be differentiated by DueDateKey, a new grouping will be defined for the result set.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
SELECT OrderDateKey, DueDateKey, SUM(SalesAmount) AS TotalSales
FROM FactInternetSalesGROUP BY OrderDateKey, DueDateKey
ORDER BY OrderDateKey;

E. Using a GROUP BY clause with a HAVING clause

The following example uses the HAVING clause to specify the groups generated in the GROUP BY clause that should be included in the result set. Only those groups with order dates in 2004 or later will be included in the results.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
SELECT OrderDateKey, SUM(SalesAmount) AS TotalSales
FROM FactInternetSales
GROUP BY OrderDateKey
HAVING OrderDateKey > 20040000
ORDER BY OrderDateKey;

See Also

Reference

SELECT (SQL Server PDW)
FROM (SQL Server PDW)


HAVING (SQL Server PDW)

[image]

HAVING (SQL Server PDW)
See Also Send Feedback

Specifies a search condition for a group or an aggregate. HAVING can be used only with the SELECT statement. HAVING is typically used in a GROUP BY clause. When GROUP BY is not used, HAVING behaves like a WHERE clause.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

[ HAVING search_condition ]

Arguments

search_condition

Specifies the search condition for the group or the aggregate to meet. For more information, see Search Condition (SQL Server PDW).

Examples

The following example uses a HAVING clause to retrieve the total for each SalesAmount from the FactInternetSales table when the OrderDateKey is in the year 2004 or later.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT OrderDateKey, SUM(SalesAmount) AS TotalSales
FROM FactInternetSales
GROUP BY OrderDateKey
HAVING SUM(SalesAmount) > 80000
ORDER BY OrderDateKey;

See Also

Reference

GROUP BY (SQL Server PDW)
WHERE (SQL Server PDW)


ORDER BY (SQL Server PDW)

[image]

ORDER BY (SQL Server PDW)
See Also Send Feedback

The ORDER BY clause specifies the sort order for columns returned in a SQL Server PDW SELECT statement.

Note Note

When ORDER BY is used in the definition of a view or subquery, the clause is used only to determine the rows returned by the TOP (SQL Server PDW) clause. The ORDER BY clause does not guarantee ordered results when these constructs are queried, unless ORDER BY is also specified in the query itself.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

[ ORDER BY
    {
    order_by_expression
    [ ASC | DESC ]
    } [ ,...n ]
]

Arguments

order_by_expression

Specifies a column on which to sort. A sort column can be specified as a name or column alias, or a positive non-negative integer representing the position of the name or alias in the select list.

ASC | DESC

Specifies that the values in the specified column should be sorted in ascending (from lowest value to highest value) or descending (from highest value to lowest value) order.

General Remarks

Null values are treated as the lowest possible values.

There is no limit to the number of items in the ORDER BY clause. However, there is a limit of 8,060 bytes for the row size of intermediate worktables needed for sort operations. This limits the total size of columns specified in an ORDER BY clause.

Multiple sort columns can be specified. The sequence of the sort columns in the ORDER BY clause defines the organization of the sorted result set. A sort column can include an expression. Column names and aliases can be qualified by the table or view name.

When the SELECT statement includes a UNION operator, the column names or column aliases in the ORDER BY clause must be included in every select list of the UNION.

Limitations and Restrictions

An integer cannot be specified when the order_by_expression appears in a ranking function.

If order_by_expression is not qualified, the value must be unique among all columns listed in the SELECT statement.

The ORDER BY clause can include items that do not appear in the select list. However, if SELECT DISTINCT is specified, if the statement contains a GROUP BY clause, or if the SELECT statement contains a UNION operator, the sort columns must appear in the select list.

Examples

The following example demonstrates ordering of a result set by the numerical EmployeeKey column in ascending order.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, FirstName, LastName FROM DimEmployee
WHERE LastName LIKE 'A%'
ORDER BY EmployeeKey;

The following example orders a result set by the numerical EmployeeKey column in descending order.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, FirstName, LastName FROM DimEmployee
WHERE LastName LIKE 'A%'
ORDER BY EmployeeKey DESC;

The following example orders a result set by the LastName column.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, FirstName, LastName FROM DimEmployee
WHERE LastName LIKE 'A%'
ORDER BY LastName;

The following example orders by two columns. This query first sorts in ascending order by the FirstName column, and then sorts common FirstName values in descending order by the LastName column.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, FirstName, LastName FROM DimEmployee
WHERE LastName LIKE 'A%'
ORDER BY LastName, FirstName;

See Also

Reference

TOP (SQL Server PDW)
SELECT (SQL Server PDW)
INSERT (SQL Server PDW)
UPDATE (SQL Server PDW)
DELETE (SQL Server PDW)


OPTION (SQL Server PDW)

[image]

OPTION (SQL Server PDW)
See Also Send Feedback

The OPTION clause specifies query options for SQL Server PDW SQL statements. The options include a query label and query join hints HASH JOIN, LOOP JOIN, MERGE JOIN, and FORCE ORDER. A query label makes it easy to locate a specific query within the SQL Server PDW Admin Console. Query join hints force SMP SQL Server, running on the Compute nodes, to use a specific join strategy when creating the SQL Server query plan.

In most cases, query join hints are not necessary because SQL Server chooses the best join strategy when it creates the query plan. However, there might be queries for which SQL Server does not generate the most optimal query execution plan, and it is for these cases that it can be beneficial to use join hints to improve query performance.

Caution note Caution

Use query join hints cautiously. Forcing SQL Server to use a suboptimal join strategy can significantly reduce query performance.

For more information about how hash join, loop join, and merge join work in SQL Server, see the following posts from Craig Freedman’s blog on MSDN:

  • Nested Loops Join

  • Hash Join

  • Merge Join

  • Summary of Join Properties

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

OPTION ( <query_option> [ ,...n ] )

<query_option> ::=
    LABEL = label_name |
    <query_hint>

<query_hint> ::=
    HASH JOIN
    | LOOP JOIN
    | MERGE JOIN
    | FORCE ORDER
    | { FORCE | DISABLE } EXTERNALPUSHDOWN

Arguments

label_name

A user-provided label name for the query, where label_name is a character string constant enclosed in single quotes. Labels identify queries in the query cache. For more information, see Constants (SQL Server PDW). The string literal can be a Unicode or non-Unicode string literal. For Unicode string literals, use of N prefix is allowed: (N’unicode_string’).

Caution note Caution

The label option is supported in PDW but is not a supported option in SQL Server.

HASH JOIN

Add the SQL Server HASH join hint to the query that will be run by SQL Server on the Compute nodes. For more information, see Hash Join on MSDN.

MERGE JOIN

Add the SQL Server MERGE join hint to the query that will be run by SQL Server on the Compute nodes. For more information, see Merge Join on MSDN.

LOOP JOIN

Add the SQL Server LOOP join hint to the Transact-SQL query that will be run by SQL Server on the Compute nodes. For more information, see Nested Loops Join on MSDN.

FORCE ORDER

Run the query by performing joins in the same order as they appear in the query.

{ FORCE | DISABLE } EXTERNALPUSHDOWN

Force or disable the pushdown of the computation of qualifying expressions or operators in Hadoop. Only applies to queries using PolyBase. Will not pushdown to Azure.

General Remarks

Query join hints are supported with the following statements:

  • SELECT (SQL Server PDW) statement

  • SELECT statement within a CREATE REMOTE TABLE AS SELECT (SQL Server PDW) statement

  • SELECT statement within a CREATE TABLE AS SELECT (SQL Server PDW) statement

  • INSERT (SQL Server PDW)

  • UPDATE (SQL Server PDW)

  • DELETE (SQL Server PDW)

When multiple query hints are specified in the same OPTION clause, SQL Server applies the hint that is the most beneficial. For example, in the OPTION clause OPTION ( HASH JOIN, LOOP JOIN, MERGE JOIN ), SQL Server will apply either the hash, loop, or merge join hint.

When the SQL Server PDW SELECT statement contains one or more subselect statements and a query hint, SQL Server, running on the Compute nodes, applies the query hint to all SELECT and subselect statements.

When a SQL Server PDW SELECT statement references a view and contains a global query hint, SQL Server PDW first expands the view and then applies the query hint when creating the parallel query plan.

[TOP]

Limitations and Restrictions

Option clause:

  • Only one OPTION clause is allowed per query.

  • The OPTION clause must be provided at the global level. For example, it cannot be appended to a subselect statement within a query.

  • When multiple SELECT statements are connected with the UNION operator, the OPTION clause can be used, but must be placed after the last SELECT statement in the UNION. Only one OPTION clause is allowed per query when the UNION operator is used.

Labels:

  • Labels cannot be used in SELECT statements that are subselects of other queries. Labels can be used to apply to entire SELECT statements that contain subselect or UNION statements.

  • Labels are only supported in SELECT, INSERT, UPDATE, and DELETE statements.

Query join hints:

  • If the query contains only a single data source, SQL Server PDW ignores query hints that exist in the query.

  • The same join hint cannot be duplicated in the OPTION clause. For example, OPTION ( HASH JOIN, HASH JOIN) is invalid.

  • CREATE VIEW (SQL Server PDW) statements cannot contain query hints. For example, if you try to create a view based on a SELECT statement that contains query hints, the CREATE VIEW statement will not compile.

[TOP]

Examples

A. SQL Server PDW SELECT statement with a label in the OPTION clause

The following example shows a simple SQL Server PDW SELECT statement with a label in the OPTION clause.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT * FROM FactResellerSales
  OPTION ( LABEL = 'q17' );

B. SQL Server PDW SELECT statement with a query hint in the OPTION clause

The following example shows a SELECT statement that uses a HASH JOIN query hint in the OPTION clause.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
SELECT COUNT (*) FROM dbo.DimCustomer a
INNER JOIN dbo.FactInternetSales b
ON (a.CustomerKey = b.CustomerKey)
OPTION (HASH JOIN);

C. SQL Server PDW SELECT statement with a label and multiple query hints in the OPTION clause

The following example is a SQL Server PDW SELECT statement that contains a label and multiple query hints. When the query is run on the Compute nodes, SQL Server will apply a hash join or merge join, according to the strategy that SQL Server decides is the most optimal.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
SELECT COUNT (*) FROM dbo.DimCustomer a
INNER JOIN dbo.FactInternetSales b
ON (a.CustomerKey = b.CustomerKey)
OPTION ( Label = 'CustJoin', HASH JOIN, MERGE JOIN);

D. Using a SQL Server PDW query hint when querying a view

The following example creates a view named CustomerView and then uses a HASH JOIN query hint in a query that references a view and a table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE VIEW CustomerView
AS
SELECT CustomerKey, FirstName, LastName FROM
    AdventureWorksPDW2012
  ..DimCustomer;

SELECT COUNT (*) FROM dbo.CustomerView a
INNER JOIN dbo.FactInternetSales b
ON (a.CustomerKey = b.CustomerKey)
OPTION (HASH JOIN);

DROP VIEW CustomerView;

E. SQL Server PDW query with a subselect and a query hint

The following example shows a query that contains both a subselect and a query hint. The query hint is applied globally. Query hints are not allowed to be appended to the subselect statement.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;
CREATE VIEW CustomerView AS
SELECT CustomerKey, FirstName, LastName FROM
    AdventureWorksPDW2012
  ..DimCustomer;

SELECT * FROM (
SELECT COUNT (*) AS a FROM dbo.CustomerView a
INNER JOIN dbo.FactInternetSales b
ON ( a.CustomerKey = b.CustomerKey )) AS t
OPTION (HASH JOIN);

F. Force the join order to match the order in the query

The following example uses the FORCE ORDER hint to force the query plan to use the join order specified by the query. This will improve performance on some queries; not all queries.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

-- Obtain partition numbers, boundary values, boundary value types, and rows per boundary
-- for the partitions in the ProspectiveBuyer table of the
    AdventureWorksPDW2012
   database.
SELECT sp.partition_number, prv.value AS boundary_value, lower(sty.name) AS boundary_value_type, sp.rows
FROM sys.tables st JOIN sys.indexes si ON st.object_id = si.object_id AND si.index_id <2
JOIN sys.partitions sp ON sp.object_id = st.object_id AND sp.index_id = si.index_id
JOIN sys.partition_schemes ps ON ps.data_space_id = si.data_space_id
JOIN sys.partition_range_values prv ON prv.function_id = ps.function_id
JOIN sys.partition_parameters pp ON pp.function_id = ps.function_id
JOIN sys.types sty ON sty.user_type_id = pp.user_type_id AND prv.boundary_id = sp.partition_number
WHERE st.object_id = (SELECT object_id FROM sys.objects WHERE name = 'FactResellerSales')
ORDER BY sp.partition_number
OPTION ( FORCE ORDER )
;

[TOP]

G. Using EXERNALPUSHDOWN

The following example forces the pushdown of the WHERE clause to the MapReduce job on the external Hadoop table.

Copy image Copy Code
SELECT ID FROM External_Table_AS A
WHERE ID < 1000000
OPTION (FORCE EXTERNALPUSHDOWN);

The following example prevents the pushdown of the WHERE clause to the MapReduce job on the external Hadoop table. All rows are returned to PDW where the WHERE clause is applied.

Copy image Copy Code
SELECT ID FROM External_Table_AS A
WHERE ID < 10
OPTION (DISABLE EXTERNALPUSHDOWN);

See Also


UNION (SQL Server PDW)

[image]

UNION (SQL Server PDW)
See Also Send Feedback

Combines the results of two or more queries into a single result set that includes all rows that belong to all queries in the union. The UNION operation is different from using joins that combine columns from two tables.

Use the following basic rules to combine the result sets of queries by using UNION:

  • The number and the order of the columns must be the same in all queries.

  • The data types must be compatible.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

    { <query_specification> | ( <query_expression> ) }
  UNION [ ALL ]
  <query_specification | ( <query_expression> )
 [ UNION [ ALL ] <query_specification> | ( <query_expression> )
    [ ...n ] ]

Arguments

<query_specification> | ( <query_expression> )

A query specification or query expression that returns data to be combined with the data from another query specification or query expression. The definitions of the columns that are part of a UNION operation do not have to be the same, but they must be compatible through implicit conversion.

ALL

Incorporates all rows into the results. This includes duplicates. If not specified, duplicate rows are removed.

General Remarks

When data types differ, the resulting data type is determined based on the rules for data type precedence.

When the data types are the same but differ in precision, scale, or length, the result is determined based on the same rules for combining expressions.

When two expressions of the same data type but different lengths are compared by using UNION, the resulting length is the length of the longer expression.

Examples

A. Using a simple UNION

In the following example, the result set includes the contents of the CustomerKey columns of both the FactInternetSales and DimCustomer tables. Since the ALL keyword is not used, duplicates are excluded from the results.

Copy image Copy Code
USE AdventureWorksPDW2012;

SELECT CustomerKey
FROM FactInternetSales
UNION
SELECT CustomerKey
FROM DimCustomer
ORDER BY CustomerKey;

B. Using UNION of two SELECT statements with ORDER BY

When any SELECT statement in a UNION statement includes an ORDER BY clause, that clause should be placed after all SELECT statements. The following example shows the incorrect and correct use of UNION in two SELECT statements in which a column is ordered with ORDER BY.

Copy image Copy Code
 USE AdventureWorksPDW2012;

-- INCORRECT
SELECT CustomerKey
FROM FactInternetSales
ORDER BY CustomerKey
UNION
SELECT CustomerKey
FROM DimCustomer
ORDER BY CustomerKey;

-- CORRECT
USE AdventureWorksPDW2012;

SELECT CustomerKey
FROM FactInternetSales
UNION
SELECT CustomerKey
FROM DimCustomer
ORDER BY CustomerKey;

C. Using UNION of two SELECT statements with WHERE and ORDER BY

The following example shows the incorrect and correct use of UNION in two SELECT statements where WHERE and ORDER BY are needed.

Copy image Copy Code
USE AdventureWorksPDW2012;

-- INCORRECT
SELECT CustomerKey
FROM FactInternetSales
WHERE CustomerKey >= 11000
ORDER BY CustomerKey
UNION
SELECT CustomerKey
FROM DimCustomer
ORDER BY CustomerKey;

-- CORRECT
USE AdventureWorksPDW2012;

SELECT CustomerKey
FROM FactInternetSales
WHERE CustomerKey >= 11000
UNION
SELECT CustomerKey
FROM DimCustomer
ORDER BY CustomerKey;

D. Using UNION of three SELECT statements to show effects of ALL and parentheses

The following examples use UNION to combine the results of the same table in order to demonstrate the effects of ALL and parentheses when using UNION.

The first example uses UNION ALL to show duplicated records and returns each row in the source table three times. The second example uses UNION without ALL to eliminate the duplicate rows from the combined results of the three SELECT statements and returns only the unduplicated rows from the source table.

The third example uses ALL with the first UNION and parentheses enclosing the second UNION that is not using ALL. The second UNION is processed first because it is in parentheses. It returns only the unduplicated rows from the table because the ALL option is not used and duplicates are removed. These rows are combined with the results of the first SELECT by using the UNION ALL keywords. This does not remove the duplicates between the two sets.

Copy image Copy Code
USE AdventureWorksPDW2012;

SELECT CustomerKey, FirstName, LastName
FROM DimCustomer
UNION ALL
SELECT CustomerKey, FirstName, LastName
FROM DimCustomer
UNION ALL
SELECT CustomerKey, FirstName, LastName
FROM DimCustomer;

SELECT CustomerKey, FirstName, LastName
FROM DimCustomer
UNION
SELECT CustomerKey, FirstName, LastName
FROM DimCustomer
UNION
SELECT CustomerKey, FirstName, LastName
FROM DimCustomer;

SELECT CustomerKey, FirstName, LastName
FROM DimCustomer
UNION ALL
(
SELECT CustomerKey, FirstName, LastName
FROM DimCustomer
UNION
SELECT CustomerKey, FirstName, LastName
FROM DimCustomer
);

See Also

Reference

SELECT (SQL Server PDW)
Expressions (SQL Server PDW)
EXCEPT and INTERSECT (SQL Server PDW)


WHERE (SQL Server PDW)

[image]

WHERE (SQL Server PDW)
See Also Send Feedback

Specifies the search condition for the rows returned by a SQL Server PDW SQLquery.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

[ WHERE search_condition ]

Arguments

search_condition

Defines the condition to be met for the rows to be returned. There is no limit to the number of predicates that can be included in a search condition. For more information about search conditions and predicates, see Search Condition (SQL Server PDW).

Examples

The following examples show how to use some common search conditions in the WHERE clause.

A. Finding a row by using a simple equality

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, LastName
FROM DimEmployee
WHERE LastName = 'Smith' ;

B. Finding rows that contain a value as part of a string

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, LastName
FROM DimEmployee
WHERE LastName LIKE ('%Smi%');

C. Finding rows by using a comparison operator

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, LastName
FROM DimEmployee
WHERE EmployeeKey  <= 500;

D. Finding rows that meet any of three conditions

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, LastName
FROM DimEmployee
WHERE EmployeeKey = 1 OR EmployeeKey = 8 OR EmployeeKey = 12;

E. Finding rows that must meet several conditions

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, LastName
FROM DimEmployee
WHERE EmployeeKey <= 500 AND LastName LIKE '%Smi%' AND FirstName LIKE '%A%';

F. Finding rows that are in a list of values

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, LastName
FROM DimEmployee
WHERE LastName IN ('Smith', 'Godfrey', 'Johnson');

G. Finding rows that have a value between two values

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, LastName
FROM DimEmployee
WHERE EmployeeKey Between 100 AND 200;

See Also

Reference

DELETE (SQL Server PDW)
SELECT (SQL Server PDW)
UPDATE (SQL Server PDW)


TOP (SQL Server PDW)

[image]

TOP (SQL Server PDW)
See Also Send Feedback

A SELECT statement clause that specifies to return only a specific number of rows from the query result.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

[
    TOP ( expression )
    [ WITH TIES ]
]

Arguments

expression

The numeric expression that specifies the number of rows to be returned. expression is implicitly converted to bigint.

If the query includes an ORDER BY clause, the top expression rows are returned as ordered by the ORDER BY clause. If the query has no ORDER BY clause, the order of the rows is arbitrary.

WITH TIES

Used when you want to return two or more rows that tie for last place in the limited results set. Must be used with the ORDER BY clause. WITH TIES may cause more rows to be returned than the value specified in expression. For example, if expression is set to 5 but 2 additional rows match the values of the ORDER BY columns in row 5, the result set will contain 7 rows.

TOP... WITH TIES can be specified only in SELECT statements, and only if an ORDER BY clause is specified. The returned order of tying records is arbitrary. ORDER BY does not affect this rule.

Examples

The following example returns the top 31 rows that match the query criteria. The ORDER BY clause is used to ensure that the 31 returned rows are the first 31 rows based on an alphabetical ordering of the LastName column.

Using TOP without specifying ties.

Copy image Copy Code
SELECT TOP (31) FirstName, LastName
FROM DimEmployee ORDER BY LastName;

Result: 31 rows are returned.

Using TOP, specifying WITH TIES.

Copy image Copy Code
SELECT TOP (31) WITH TIES FirstName, LastName
FROM DimEmployee ORDER BY LastName;

Result: 33 rows are returned, because 3 employees named Brown tie for the 31st row.

See Also

Reference

SELECT (SQL Server PDW)


Aliasing (SQL Server PDW)

[image]

Aliasing (SQL Server PDW)
See Also Send Feedback

Aliasing allows the temporary substitution of a short and easy-to-remember string in place of a table or column name in SQL Server PDW SQL queries. Table aliases are often used in JOIN queries because the JOIN syntax requires fully qualified object names when referencing columns.

Aliases must be single words conforming to object naming rules. For more information, see Object Naming Rules (SQL Server PDW). Aliases cannot contain blank spaces and cannot be enclosed in either single or double quotes.

Syntax

object_source [AS ] alias

Arguments

object_source

The name of the source table or column.

AS

An optional alias preposition. When working with range variable aliasing, the AS keyword is prohibited.

alias

The desired temporary reference name for the table or column. Any valid object name can be used. For more information, see Object Naming Rules (SQL Server PDW).

Examples

The following example shows a query with multiple joins. Both table and column aliasing are demonstrated in this example.

  • Column Aliasing: Both columns and expressions involving columns in the select list are aliased in this example. SalesTerritoryRegion AS SalesTR demonstrates a simple column alias. Sum(SalesAmountQuota) AS TotalSales demonstrates

  • Table Aliasing: dbo.DimSalesTerritory AS st shows creation of the st alias for the dbo.DimSalesTerritory table.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT LastName, SUM(SalesAmountQuota) AS TotalSales, SalesTerritoryRegion AS SalesTR,
    RANK() OVER (PARTITION BY SalesTerritoryRegion ORDER BY SUM(SalesAmountQuota) DESC ) AS RankResult
FROM dbo.DimEmployee AS e
INNER JOIN dbo.FactSalesQuota AS sq ON e.EmployeeKey = sq.EmployeeKey
INNER JOIN dbo.DimSalesTerritory AS st ON e.SalesTerritoryKey = st.SalesTerritoryKey
WHERE SalesPersonFlag = 1 AND SalesTerritoryRegion != N'NA'
GROUP BY LastName, SalesTerritoryRegion;

The AS keyword can be excluded, as shown below, but is often included for readability.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT LastName, SUM(SalesAmountQuota) TotalSales, SalesTerritoryRegion SalesTR,
RANK() OVER (PARTITION BY SalesTerritoryRegion ORDER BY SUM(SalesAmountQuota) DESC ) RankResult
FROM dbo.DimEmployee e
INNER JOIN dbo.FactSalesQuota sq ON e.EmployeeKey = sq.EmployeeKey
INNER JOIN dbo.DimSalesTerritory st ON e.SalesTerritoryKey = st.SalesTerritoryKey
WHERE SalesPersonFlag = 1 AND SalesTerritoryRegion != N'NA'
GROUP BY LastName, SalesTerritoryRegion;

See Also

Reference

SELECT (SQL Server PDW)
INSERT (SQL Server PDW)
UPDATE (SQL Server PDW)


Search Condition (SQL Server PDW)

[image]

Search Condition (SQL Server PDW)
See Also Send Feedback

The search condition, in SQL Server PDW, is a combination of one or more predicates that use the logical operators AND, OR, and NOT.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

< search_condition > ::=
    { [ NOT ] <predicate> | ( <search_condition> ) }
    [ { AND | OR } [ NOT ] { <predicate> | ( <search_condition> ) } ]
[ ,...n ]

<predicate> ::=
    { expression { = | < > | ! = | > | > = | < | < = } expression
    | string_expression [ NOT ] LIKE string_expression
    | expression [ NOT ] BETWEEN expression AND expression
    | expression IS [ NOT ] NULL
    | expression [ NOT ] IN ( subquery | expression [ ,...n ] )
    | expression [ NOT ] EXISTS ( subquery )     }

Arguments

<search_condition>

Specifies the conditions for the rows returned in the result set for a SELECT statement, query expression, or subquery. For an UPDATE statement, specifies the rows to be updated. For a DELETE statement, specifies the rows to be deleted. There is no limit to the number of predicates that can be included in a SQL statement search condition.

NOT

Negates the Boolean expression specified by the predicate. For more information, see NOT (SQL Server PDW).

AND

Combines two conditions and evaluates to TRUE when both of the conditions are TRUE. For more information, see AND (SQL Server PDW).

OR

Combines two conditions and evaluates to TRUE when either condition is TRUE. For more information, see OR (SQL Server PDW).

< predicate >

An expression that returns TRUE, FALSE, or UNKNOWN.

expression

A column name, a constant, a function, a variable, a scalar subquery, or any combination of column names, constants, and functions connected by an operator(s) or a subquery. The expression can also contain the CASE expression.

=

The operator used to test the equality between two expressions. For more information, see = (Equals) (SQL Server PDW).

<>

The operator used to test the condition of two expressions not being equal to each other. For more information, see <> (Not Equal To) (SQL Server PDW).

!=

The operator used to test the condition of two expressions not being equal to each other. For more information, see != (Not Equal To) (SQL Server PDW).

>

The operator used to test the condition of one expression being greater than the other. For more information, see > (Greater Than) (SQL Server PDW).

>=

The operator used to test the condition of one expression being greater than or equal to the other expression. For more information, see >= (Greater Than or Equal To) (SQL Server PDW).

<

The operator used to test the condition of one expression being less than the other. For more information, see < (Less Than) (SQL Server PDW).

<=

The operator used to test the condition of one expression being less than or equal to the other expression. For more information, see <= (Less Than or Equal To) (SQL Server PDW).

string_expression

A string of characters and wildcard characters.

[ NOT ] LIKE

Indicates that the subsequent character string is to be used with pattern matching. For more information, see LIKE (SQL Server PDW).

[ NOT ] BETWEEN

Specifies an inclusive range of values. Use AND to separate the starting and ending values. For more information, see BETWEEN (SQL Server PDW).

IS [ NOT ] NULL

Specifies a search for null values, or for values that are not null, depending on the keywords used. An expression with a bitwise or arithmetic operator evaluates to NULL if any one of the operands is NULL. For more information, see IS [NOT] NULL (SQL Server PDW).

[ NOT ] IN

Specifies the search for an expression, based on whether the expression is included in or excluded from a list. The search expression can be a constant or a column name. The list can be a set of constants or, more typically, a subquery. Enclose the list of values in parentheses. For more information, see IN (SQL Server PDW).

subquery

Can be considered a restricted SELECT statement and is similar to <query_expression> in the SELECT statement. The ORDER BY clause, the COMPUTE clause, and the INTO keyword are not allowed. For more information, see SELECT (SQL Server PDW).

EXISTS

Used with a subquery to test for the existence of rows returned by the subquery. For more information, see EXISTS (SQL Server PDW).

General Remarks

The order of precedence for the logical operators from highest to lowest is: NOT, AND, OR. Use parantheses to override this precedence in a search condition.

The order of evaluation of logical operators can vary, depending on choices made by the query optimizer. For more information, see NOT (SQL Server PDW), AND (SQL Server PDW), and OR (SQL Server PDW).

Remarks

Aggregate functions are not supported in WHERE or HAVING clauses.

Examples

A. Using WHERE with LIKE

The following example searches for the rows in which the LastName column has the characters and.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, LastName
FROM DimEmployee
WHERE LastName LIKE '%and%';

B. Using WHERE and LIKE syntax with Unicode data

The following example uses the WHERE clause to perform a Unicode search on the LastName column.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

SELECT EmployeeKey, LastName
FROM DimEmployee
WHERE LastName LIKE N'%and%';

See Also

Reference

CASE (SQL Server PDW)
DELETE (SQL Server PDW)
Expressions (SQL Server PDW)
FROM (SQL Server PDW)
Operators (SQL Server PDW)
UPDATE (SQL Server PDW)


Subqueries (SQL Server PDW)

[image]

Subqueries (SQL Server PDW)
See Also Send Feedback

This topic gives examples of using subqueries in SQL Server PDW.

For the SELECT statement, see SELECT (SQL Server PDW)

Contents

  • Basics

  • Examples

Basics

Subquery

A subquery is a query that is nested inside a SELECT, INSERT, UPDATE, or DELETE statement, or inside another subquery. This is also called an inner query or inner select.

Outer query

The statement that contains the subquery. This is also called an outer select.

Correlated subquery

A subquery that refers to a table in the outer query.

[TOP]

Examples

This section provides examples of subqueries supported in SQL Server PDW.

A. TOP and ORDER BY in a subquery

Copy image Copy Code
SELECT * FROM tblA
WHERE col1 IN
    (SELECT TOP 100 col1 FROM tblB ORDER BY col1);

B. HAVING clause with a correlated subquery

Copy image Copy Code
SELECT dm.EmployeeKey, dm.FirstName, dm.LastName
FROM DimEmployee AS dm
GROUP BY dm.EmployeeKey, dm.FirstName, dm.LastName
HAVING 5000 <=
(SELECT sum(OrderQuantity)
FROM FactResellerSales AS frs
WHERE dm.EmployeeKey = frs.EmployeeKey)
ORDER BY EmployeeKey;

C. Correlated subqueries with analytics

Copy image Copy Code
SELECT * FROM ReplA AS A
WHERE A.ID IN
    (SELECT sum(B.ID2) OVER() FROM ReplB AS B WHERE A.ID2 = B.ID);

D. Correlated union statements in a subquery

Copy image Copy Code
SELECT * FROM RA
WHERE EXISTS
    (SELECT 1 FROM RB WHERE RB.b1 = RA.a1
     UNION ALL SELECT 1 FROM RC);

E. Join predicates in a subquery

Copy image Copy Code
SELECT * FROM RA INNER JOIN RB
    ON RA.a1 = (SELECT COUNT(*) FROM RC);

F. Correlated join predicates in a subquery

Copy image Copy Code
SELECT * FROM RA
    WHERE RA.a2 IN
    (SELECT 1 FROM RB INNER JOIN RC ON RA.a1=RB.b1+RC.c1);

G. Correlated subselects as data sources

Copy image Copy Code
SELECT * FROM RA
    WHERE 3 = (SELECT COUNT(*)
        FROM (SELECT b1 FROM RB WHERE RB.b1 = RA.a1) X);

H. Correlated subqueries in the data values used with aggregates

Copy image Copy Code
SELECT Rb.b1, (SELECT RA.a1 FROM RA WHERE RB.b1 = RA.a1) FROM RB GROUP BY RB.b1;

I. Using IN with a correlated subquery

The following example uses IN in a correlated, or repeating, subquery. This is a query that depends on the outer query for its values. The inner query is run repeatedly, one time for each row that may be selected by the outer query. This query retrieves one instance of the EmployeeKey plus first and last name of each employee for which the OrderQuantity in the FactResellerSales table is 5 and for which the employee identification numbers match in the DimEmployee and FactResellerSales tables.

Copy image Copy Code
SELECT DISTINCT dm.EmployeeKey, dm.FirstName, dm.LastName
FROM DimEmployee AS dm
WHERE 5 IN
    (SELECT OrderQuantity
    FROM FactResellerSales AS frs
    WHERE dm.EmployeeKey = frs.EmployeeKey)
ORDER BY EmployeeKey;

J. Using EXISTS versus IN with a subquery

The following example shows queries that are semantically equivalent to illustrate the difference between using the EXISTS keyword and the IN keyword. Both are examples of a subquery that retrieves one instance of each product name for which the product subcategory is Road Bikes. ProductSubcategoryKey matches between the DimProduct and DimProductSubcategory tables.

Copy image Copy Code
SELECT DISTINCT EnglishProductName
FROM DimProduct AS dp
WHERE EXISTS
    (SELECT *
     FROM DimProductSubcategory AS dps
     WHERE dp.ProductSubcategoryKey = dps.ProductSubcategoryKey
           AND dps.EnglishProductSubcategoryName = 'Road Bikes')
ORDER BY EnglishProductName;

Or

Copy image Copy Code
SELECT DISTINCT EnglishProductName
FROM DimProduct AS dp
WHERE dp.ProductSubcategoryKey IN
    (SELECT ProductSubcategoryKey
     FROM DimProductSubcategory
     WHERE EnglishProductSubcategoryName = 'Road Bikes')
ORDER BY EnglishProductName;

K. Using multiple correlated subqueries

This example uses two correlated subqueries to find the names of employees who have sold a particular product.

Copy image Copy Code
SELECT DISTINCT LastName, FirstName, e.EmployeeKey
FROM DimEmployee e JOIN FactResellerSales s ON e.EmployeeKey = s.EmployeeKey
WHERE ProductKey IN
(SELECT ProductKey FROM DimProduct WHERE ProductSubcategoryKey IN
(SELECT ProductSubcategoryKey FROM DimProductSubcategory
 WHERE EnglishProductSubcategoryName LIKE '%Bikes'))
ORDER BY LastName
;

[TOP]

See Also


Security Statements (SQL Server PDW)

[image]

Security Statements (SQL Server PDW)
See Also Send Feedback

SQL Server PDW includes statements for manipulation of logins, users, roles, security related objects.

This topic is organized by object type, as shown:

Login and Server Role Management

Master Keys, Database Keys, and Certificates

Special Statements

User, Login, and Role Management

Logins | Server Roles | Users | Database Roles
CREATE LOGIN | CREATE SERVER ROLE | CREATE USER | CREATE ROLE
ALTER LOGIN | ALTER SERVER ROLE | ALTER USER | ALTER ROLE
DROP LOGIN | DROP SERVER ROLE | DROP USER | DROP ROLE

Master Keys, Database Keys, and Certificates

Master Keys | Database Encryption Keys | Certificates
CREATE MASTER KEY | CREATE DATABASE ENCRYPTION KEY | CREATE CERTIFICATE
ALTER MASTER KEY | ALTER DATABASE ENCRYPTION KEY | BACKUP CERTIFICATE
OPEN MASTER KEY | DROP DATABASE ENCRYPTION KEY | ALTER CERTIFICATE
CLOSE MASTER KEY |  | DROP CERTIFICATE
DROP MASTER KEY |  |

Special Statements

Command | Description
ALTER AUTHORIZATION | Changes ownership of an object.
Permissions: GRANT, DENY, REVOKE | Describes how to manage object permissions.

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Definition Statements (SQL Server PDW)
Data Manipulation Statements (SQL Server PDW)
Queries (SQL Server PDW)


Permissions: GRANT, DENY, REVOKE (SQL Server PDW)

[image]

Permissions: GRANT, DENY, REVOKE (SQL Server PDW)
See Also Send Feedback

Use SQL Server PDW GRANT and DENY statements to grant or deny a permission (such as UPDATE) on a securable (such as a database, table, view, etc.) to a security principal (a login, a database user, or a database role). Use REVOKE to remove the grant or deny of a permission.

Server level permissions are applied to logins. Database level permissions are applied to database users and database roles.

To see what permissions have been granted and denied, query the sys.server_permissions and sys.database_permissions views. Permissions that are not explicitly granted or denied to a security principal can be inherited by having membership in a role that has permissions. The permissions of the fixed database roles cannot be changed and do not appear in the sys.server_permissions and sys.database_permissions views.

  • GRANT explicitly grants one or more permissions.

  • DENY explicitly denies the principal from having one or more permissions.

  • REVOKE removes existing GRANT or DENY permissions.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

GRANT
    <permission> [ ,...n ]
    [ ON [ <class_type> :: ] securable ]
    TO principal [ ,...n ]
    [ WITH GRANT OPTION ]
[;]

DENY
    <permission> [ ,...n ]
    [ ON [ <class_type> :: ] securable ]
    TO principal [ ,...n ]
    [ CASCADE ]
[;]

REVOKE
    <permission> [ ,...n ]
    [ ON [ <class_type> :: ] securable ]
    [ FROM | TO ] principal [ ,...n ]
    [ CASCADE ]
[;]

<permission> ::=
{ see the tables below }

<class_type> ::=
{
      LOGIN
    | DATABASE
    | OBJECT
    | ROLE
    | SCHEMA
    | USER
}

Arguments

<permission>[ ,...n ]

One or more permissions to grant, deny, or revoke.

ON [ <class_type> :: ] securable

The ON clause describes the securable parameter on which to grant, deny, or revoke permissions.

<class_type>

The class type of the securable. This can be LOGIN, DATABASE, OBJECT, SCHEMA, ROLE, or USER. Permissions can also be granted to the SERVER class_type, but SERVER is not specified for those permissions. DATABASE is not specified when the permission includes the word DATABASE (for example ALTER ANY DATABASE). When no class_type is specified and the permission type is not restricted to the server or database class, the class is assumed to be OBJECT.

securable

The name of the login, database, table, view, schema, procedure, role, or user on which to grant, deny, or revoke permissions. The object name can be specified with the three-part naming rules that are described in Syntax Conventions (SQL Server PDW).

TO principal [ ,...n ]

One or more principals being granted, denied, or revoked permissions. Principal is the name of a login, database user, or database role.

FROM principal [ ,...n ]

One or more principals to revoke permissions from. Principal is the name of a login, database user, or database role. FROM can only be used with a REVOKE statement. TO can be used with GRANT, DENY, or REVOKE.

WITH GRANT OPTION

Indicates that the grantee will also be given the ability to grant the specified permission to other principals.

CASCADE

Indicates that the permission is denied or revoked to the specified principal and to all other principals to which the principal granted the permission. Required when the principal has the permission with GRANT OPTION.

GRANT OPTION FOR

Indicates that the ability to grant the specified permission will be revoked. This is required when you are using the CASCADE argument.

Important note Important

If the principal has the specified permission without the GRANT option, the permission itself will be revoked.

Permissions

To grant a permission, the grantor must have either the permission itself with the WITH GRANT OPTION, or must have a higher permission that implies the permission being granted. Object owners can grant permissions on the objects they own. Principals with CONTROL permission on a securable can grant permission on that securable. Members of the db_owner and db_securityadmin fixed database roles can grant any permission in the database.

General Remarks

Denying or revoking permissions to a principal will not affect requests that have passed authorization and are currently running. To restrict access immediately, you must cancel active requests or kill current sessions.

Note Note

Most fixed server roles are not available in this release. Use user-defined database roles instead. Logins cannot be added to the sysadmin fixed server role. Granting the CONTROL SERVER permission approximates membership in the sysadmin fixed server role. For more information, see Fixed Server Roles (SQL Server PDW).

Some statements require multiple permissions. For example, to create a table requires the CREATE TABLE permissions in the database, and the ALTER SCHEMA permission for the table that will contain the table.

For specific examples of setting permissions, see the topic PDW Permissions (SQL Server PDW) and its subordinate topics.

PDW sometimes executes stored procedures to distribute user actions to the compute nodes. Therefore, the execute permission for an entire database cannot be denied. (For example DENY EXECUTE ON DATABASE::<name> TO <user>; will fail.) As a work around, deny the execute permission to user-schemas or specific objects (procedures).

Implicit and Explicit Permissions

An explicit permission is a GRANT or DENY permission given to a principal by a GRANT or DENY statement.

An implicit permission is a GRANT or DENY permission that a principal (login, user, or database role) has inherited from another database role.

An implicit permission can also be inherited from a covering or parent permission. For example, UPDATE permission on a table can be inherited by having UPDATE permission on the schema that contains the table, or CONTROL permission on the table.

Ownership Chaining

When multiple database objects access each other sequentially, the sequence is known as a chain. Although such chains do not independently exist, when SQL Server traverses the links in a chain, SQL Server evaluates permissions on the constituent objects differently than it would if it were accessing the objects separately. Ownership chaining has important implications for managing security. For more information about ownership chains, see Ownership Chains and Tutorial: Ownership Chains and Context Switching.

Permission List

Server Level Permissions

Server level permissions can be granted, denied, and revoked from logins.

Permissions that apply to servers

  • CONTROL SERVER

  • ADMINISTER BULK OPERATIONS

  • ALTER ANY CONNECTION

  • ALTER ANY DATABASE

  • CREATE ANY DATABASE

  • ALTER ANY EXTERNAL DATA SOURCE

  • ALTER ANY EXTERNAL FILE FORMAT

  • ALTER ANY LOGIN

  • ALTER SERVER STATE

  • CONNECT SQL

  • VIEW ANY DEFINITION

  • VIEW ANY DATABASE

  • VIEW SERVER STATE

Permissions that apply to logins

  • CONTROL ON LOGIN

  • ALTER ON LOGIN

  • IMPERSONATE ON LOGIN

  • VIEW DEFINITION

Database Level Permissions

Database level permissions can be granted, denied, and revoked from database users and user-defined database roles.

Permissions that apply to all database classes

  • CONTROL

  • ALTER

  • VIEW DEFINITION

Permissions that apply to all database classes except users

  • TAKE OWNERSHIP

Permissions that apply only to databases

  • ALTER ANY DATABASE

  • ALTER ON DATABASE

  • ALTER ANY DATASPACE

  • ALTER ANY ROLE

  • ALTER ANY SCHEMA

  • ALTER ANY USER

  • BACKUP DATABASE

  • CONNECT ON DATABASE

  • CREATE PROCEDURE

  • CREATE ROLE

  • CREATE SCHEMA

  • CREATE TABLE

  • CREATE VIEW

  • SHOWPLAN

Permissions that apply only to users

  • IMPERSONATE

Permissions that apply to databases, schemas, and objects

  • ALTER

  • DELETE

  • EXECUTE

  • INSERT

  • SELECT

  • UPDATE

  • REFRENCES

For a definition of each type of permission, see Permissions (Database Engine).

Chart of Permissions

All permissions are graphically represented on this poster. This is the easiest way to see nested hierarchy of permissions. For example the ALTER ON LOGIN permission can be granted by itself, but it is also included if a login is granted the CONTROL permission on that login, or if a login is granted the ALTER ANY LOGIN permission.

APS security permissions poster

To download a full size version of this poster, see SQL Server PDW Permissions in the files section of the APS Yammer site (or request by e-mail from apsdoc@microsoft.com.

Default Permissions

The following list describes the default permissions:

  • When a login is created by using the CREATE LOGIN statement the new login receives the CONNECT SQL permission.

  • All logins are members of the public server role and cannot be removed from public.

  • When a database user is created by using the CREATE USER permission, the database user receives the CONNECT permission in the database.

  • All principals, including the public role, have no explicit or implicit permissions by default.

  • When a login or user becomes the owner of a database or object, the login or user always has all permissions on the database or object. The ownership permissions cannot be changed and are not visible as explicit permissions. The GRANT, DENY, and REVOKE statements have no effect on owners.

  • The sa login has all permissions on the appliance. Similar to ownership permissions, the sa permissions cannot be changed and are not visible as explicit permissions. The GRANT, DENY, and REVOKE statements have no effect on sa login. The sa login cannot be renamed.

  • The USE statement does not require permissions. All principals can run the USE statement on any database.

Examples

A. Granting a server level permission to a login

The following two statements grant a server level permission to a login.

Copy image Copy Code
GRANT CONTROL SERVER TO [Ted];
Copy image Copy Code
GRANT ALTER ANY DATABASE TO Mary;

B. Granting a server level permission to a login

The following example grants a server level permission on a login to a server principal (another login).

Copy image Copy Code
GRANT  VIEW DEFINITION ON LOGIN::Ted TO Mary;

C. Granting a database level permission to a user

The following example grants a database level permission on a user to a database principal (another user).

Copy image Copy Code
GRANT VIEW DEFINITION ON USER::[Ted] TO Mary;

D. Granting, denying, and revoking a schema permission

The following GRANT statement grants Yuen the ability to select data from any table or view in the dbo schema.

Copy image Copy Code
GRANT SELECT ON SCHEMA::dbo TO [Yuen];

The following DENY statement prevents Yuen from selecting data from any table or view in the dbo schema. Yuen cannot read the data even if he has permission in some other way, such as through a role membership.

Copy image Copy Code
DENY SELECT ON SCHEMA::dbo TO [Yuen];

The following REVOKE statement removes the DENY permission. Now Yuen's explicit permissions are neutral. Yuen might be able to select data from any table through some other implicit permission such as a role membership.

Copy image Copy Code
REVOKE SELECT ON SCHEMA::dbo TO [Yuen];

E. Demonstrating the optional OBJECT:: clause

Because OBJECT is the default class for a permission statement, the following two statements are the same. The OBJECT:: clause is optional.

Copy image Copy Code
GRANT UPDATE ON OBJECT::dbo.StatusTable TO [Ted];
Copy image Copy Code
GRANT UPDATE ON dbo.StatusTable TO [Ted];

See Also


ALTER AUTHORIZATION (SQL Server PDW)

[image]

ALTER AUTHORIZATION (SQL Server PDW)
See Also Send Feedback

Changes the owner of a database, schema, owner or table owner to a different SQL Server PDW security principal. Use this statement when managing security on databases and tables.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER AUTHORIZATION ON
    [ <class_type> :: ] <entity_name>
    TO { principal_name | SCHEMA OWNER }
[;]

<class_type> ::=
{
    DATABASE
    | SCHEMA
    | OBJECT
}

<entity_name> ::=
{
    database_name
    | schema_name
    | [ schema_name. ] object_name
}

Arguments

<class_type>

The class of the securable that is changing ownership. The class is either DATABASE, SCHEMA, or OBJECT. The default is OBJECT. Using OBJECT is optional since it is the default. If the class type is DATABASE or SCHEMA, it must be specified. The object can be a table, view, or procedure.

<entity_name>

The name of the entity that will have the ownership change. The entity is either a database or a table.

database_name

The name of the database that will have the ownership change.

[ schema_name. ] object_name

The one or two-part name for the table, view, or procedure that will have the ownership change.

principal_name | SCHEMA OWNER

Name of the security principal that will own the entity. Database objects must be owned by a database principal; a database user or role. Server objects (such as databases) must be owned by a server principal (a login). Specify SCHEMA OWNER as the principal_name to indicate that the object should be owned by the principal that owns the schema of the object.

Permissions

Requires TAKE OWNERSHIP permission on the entity. If the new owner is not the user that is executing this statement, also requires either, 1) IMPERSONATE permission on the new owner if it is a user or login; or 2) if the new owner is a role, membership in the role, or ALTER permission on the role;

Examples

A. Changing the owner of a table

Each of the following examples changes the owner of the Sprockets table in the Parts database to the database user MichikoOsada.

Copy image Copy Code
ALTER AUTHORIZATION ON Sprockets TO MichikoOsada;
ALTER AUTHORIZATION ON dbo.Sprockets TO MichikoOsada;
ALTER AUTHORIZATION ON OBJECT::Sprockets TO MichikoOsada;
ALTER AUTHORIZATION ON OBJECT::dbo.Sprockets TO MichikoOsada;

B. Changing the owner of a database

The following example change the owner of the Parts database to the login MichikoOsada.

Copy image Copy Code
ALTER AUTHORIZATION ON DATABASE::Parts TO MichikoOsada;

See Also

Reference

Permissions: GRANT, DENY, REVOKE (SQL Server PDW)


ALTER CERTIFICATE (SQL Server PDW)

[image]

ALTER CERTIFICATE (SQL Server PDW)
See Also Send Feedback

Removes the private key used to encrypt a certificate, or adds one if none is present. Can only be executed in the master database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER CERTIFICATE certificate_name
{
    REMOVE PRIVATE KEY
    |
    WITH PRIVATE KEY (
        FILE = '<path_to_private_key>',
        DECRYPTION BY PASSWORD = '<key password>'
    )}

Arguments

certificate_name

Is the unique name by which the certificate is known in database.

FILE ='path_to_private_key'

Specifies the complete path, including file name, to the private key. This parameter must be a UNC path to a network location. This file will be accessed within the security context of a credential added by using sp_pdw_add_network_credential. When you use this option, you must make sure that the credential account has access to the specified file.

DECRYPTION BY PASSWORD ='key_password'

Specifies the password that is required to decrypt the private key.

Security note Security Note

Communication traffic between components inside the SQL Server PDW is not encrypted. High privileged users could potentially install network sniffing software to obtain the certificate private and public keys. Mitigate this risk by controlling access to the SQL Server PDW.

REMOVE PRIVATE KEY

Specifies that the private key should no longer be maintained inside the database.

Remarks

The private key must correspond to the public key specified by certificate_name.

When the private key of a certificate that already exists in the database is imported from a file, the private key will be automatically protected by the database master key.

The REMOVE PRIVATE KEY option will delete the private key of the certificate from the database. Do not remove the private key of a certificate that protects a symmetric key.

Important note Important

Always make an archival copy of a private key before removing it from a database. For more information, see BACKUP CERTIFICATE (SQL Server PDW).

Permissions

Requires CONTROL SERVER permission.

Examples

A. Importing a private key for a certificate that is already present in the database

Copy image Copy Code
ALTER CERTIFICATE Shipping13
    WITH PRIVATE KEY (FILE = '\\ServerA7\importedkeys\Shipping13',
    DECRYPTION BY PASSWORD = 'GDFLKl8^^GGG4000%');
GO

See Also

Reference

CREATE CERTIFICATE (SQL Server PDW)
DROP CERTIFICATE (SQL Server PDW)
BACKUP CERTIFICATE (SQL Server PDW)
sys.certificates (SQL Server PDW)


ALTER DATABASE ENCRYPTION KEY (SQL Server PDW)

[image]

ALTER DATABASE ENCRYPTION KEY (SQL Server PDW)
See Also Send Feedback

Alters an encryption key and certificate that is used for transparently encrypting a database. For more information about transparent database encryption, see Transparent Data Encryption (SQL Server PDW).

Caution note Caution

Regenerating the database encryption key requires decrypting and then re-encrypting the entire database. Changing the certificate occurs quickly because it only triggers re-encryption of the DEK that is protected by a certificate.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER DATABASE ENCRYPTION KEY
    {
      {
        REGENERATE WITH ALGORITHM = { AES_128 | AES_192 | AES_256 | TRIPLE_DES_3KEY }
        [ ENCRYPTION BY SERVER CERTIFICATE Encryptor_Name ]
      }
      |
      ENCRYPTION BY SERVER   CERTIFICATE Encryptor_Name
    }
[ ; ]

Arguments

REGENERATE WITH ALGORITHM = { AES_128 | AES_192 | AES_256 | TRIPLE_DES_3KEY }

Specifies the encryption algorithm that is used for the encryption key.

ENCRYPTION BY SERVER CERTIFICATE Encryptor_Name

Specifies the name of the certificate used to encrypt the database encryption key.

ENCRYPTION BY SERVER ASYMMETRIC KEY is not supported in this version of SQL Server PDW.

Remarks

The certificate that is used to encrypt the database encryption key must be located in the master database.

The database encryption key does not have to be regenerated when a database owner (dbo) is changed.

Permissions

Requires CONTROL SERVER permission.

Examples

The following example alters the database encryption key to use the AES_256 algorithm.

Copy image Copy Code
USE AdventureWorks2012;
GO
ALTER DATABASE ENCRYPTION KEY
REGENERATE WITH ALGORITHM = AES_256;
GO

See Also

Reference

CREATE DATABASE ENCRYPTION KEY (SQL Server PDW)
DROP DATABASE ENCRYPTION KEY (SQL Server PDW)
sys.dm_pdw_nodes_database_encryption_keys (SQL Server PDW)
sp_pdw_database_encryption_regenerate_system_keys (SQL Server PDW)

Concepts

Transparent Data Encryption (SQL Server PDW)


ALTER LOGIN (SQL Server PDW)

[image]

ALTER LOGIN (SQL Server PDW)
See Also Send Feedback

Changes the properties of a login account.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER LOGIN login_name
    {
    <status_option>
    | WITH <set_option> [ ,... ]
    }

<status_option> ::=
        ENABLE | DISABLE

<set_option> ::=
    PASSWORD = 'password'
    [
      OLD_PASSWORD = 'oldpassword'
      | <password_option> [<password_option> ]
    ]
    | NAME = login_name
    | CHECK_POLICY = { ON | OFF }
    | CHECK_EXPIRATION = { ON | OFF }
<password_option> ::=
    MUST_CHANGE | UNLOCK

Arguments

login_name

Specifies the name of the login that is being changed.

ENABLE | DISABLE

Enables or disables this login. Disabling a login does not affect the behavior of logins that are already connected. Disabled logins retain their permissions and can still be impersonated.

PASSWORD ='password'

Applies only to SQL Server authentication logins. Specifies the new password for the login that is being changed. Passwords are case-sensitive. Passwords should always be at least 8 characters long, and cannot exceed 128 characters. Passwords can include a-z, A-Z, 0-9, and most non-alphanumeric characters. Passwords cannot contain single quotes, or the login_name. Additionally, passwords must contain at least three of the following:

  • An uppercase character (A-Z).

  • A lowercase character (a-z).

  • A digit (0-9).

  • One of the non-alphanumeric characters, such as a space, _, @, *, ^, %, !, $, #, or &.

Caution note Caution

The characters ^ % & are permitted in passwords, however PowerShell regards these as special characters. If these characters are used in passwords for the system administrator or SQL Server sa accounts (the AdminPassword and PdwSAPassword parameters during setup) then setup, including INSTALL, UPGRADE, REPLACENODE, and PATCHING, will fail. To ensure a successful upgrade when current passwords contain unsupported characters, change these passwords so that they do not contain such characters before running upgrade. After upgrade completes, you can set these passwords back to their original values.

OLD_PASSWORD ='oldpassword'

Applies only to SQL Server authentication logins. The current password of the login to which a new password will be assigned. Passwords are case-sensitive.

MUST_CHANGE

Applies only to SQL Server authentication logins. If this option is included, SQL Server will prompt for an updated password the first time the altered login is used.

NAME = login_name

The new name of the login that is being renamed. The new name of a login must conform to the Object Naming Rules (SQL Server PDW). If this is a Windows login, the SID of the Windows principal corresponding to the new name must match the SID associated with the login in SQL Server. The new name of a SQL Server login cannot contain a backslash character (\).

CHECK_EXPIRATION = { ON | OFF }

Applies only to SQL Server authentication logins. Specifies whether password expiration policy should be enforced on this login.

CHECK_POLICY = { ON | OFF }

Applies only to SQL Server authentication logins. Specifies that the Windows password policies of the SQL Server PDW system on which SQL Server is running should be enforced on this login.

UNLOCK

Applies only to SQL Server authentication logins. Specifies that a login that is locked out should be unlocked.

General Remarks

When CHECK_POLICY is changed to ON, the following behavior occurs:

  • The password history is initialized with the value of the current password hash.

When CHECK_POLICY is changed to OFF, the following behavior occurs:

  • CHECK_EXPIRATION is also set to OFF.

  • The password history is cleared.

  • The value of lockout_time is reset.

If MUST_CHANGE is specified, CHECK_EXPIRATION and CHECK_POLICY must be set to ON. Otherwise, the statement will fail.

If CHECK_POLICY is set to OFF, CHECK_EXPIRATION cannot be set to ON. An ALTER LOGIN statement that has this combination of options will fail.

When a password has expired, the login can log in, but cannot perform any actions other than changing the password with the ALTER LOGIN statement.

For a list of policies inherited from Windows, see CREATE LOGIN (SQL Server PDW).

You cannot use ALTER_LOGIN with the DISABLE argument to deny access to a Windows group. For example, ALTER_LOGIN [domain\group] DISABLE will return the following error message:

"Msg 15151, Level 16, State 1, Line 1

"Cannot alter the login 'Domain\Group', because it does not exist or you do not have permission."

This is by design.

Permissions

Requires ALTER permission on the login. A principal can change the password and default datase for its own login.

Examples

A. Enabling a disabled login

The following example enables the login Mary5.

Copy image Copy Code
ALTER LOGIN Mary5 ENABLE;

B. Changing the password of a login

The following example changes the password of login Mary5 to a strong password.

Copy image Copy Code
ALTER LOGIN Mary5 WITH PASSWORD = '<enterStrongPasswordHere>';

C. Unlocking a login

The following example unlocks the login Mary5.

Copy image Copy Code
ALTER LOGIN Mary8 WITH PASSWORD = 'jlkfdjl;ied51' MUST_CHANGE UNLOCK;

To unlock a login without changing the password, turn the check policy off and then on again.

Copy image Copy Code
ALTER LOGIN [Mary5] WITH CHECK_POLICY = OFF;
ALTER LOGIN [Mary5] WITH CHECK_POLICY = ON;
GO

C. Changing the name of a login

The following example changes the name of login Mary5 to John2.

Copy image Copy Code
ALTER LOGIN Mary5 WITH NAME = John2;

See Also

Reference

CREATE LOGIN (SQL Server PDW)
DROP LOGIN (SQL Server PDW)


ALTER MASTER KEY (SQL Server PDW)

[image]

ALTER MASTER KEY (SQL Server PDW)
See Also Send Feedback

Changes the properties of a database master key.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER MASTER KEY <alter_option>

<alter_option> ::=
    <regenerate_option> | <encryption_option>

<regenerate_option> ::=
    [ FORCE ] REGENERATE WITH ENCRYPTION BY PASSWORD = 'password'

<encryption_option> ::=
    ADD ENCRYPTION BY SERVICE MASTER KEY
    |
    DROP ENCRYPTION BY SERVICE MASTER KEY

Arguments

PASSWORD ='password'

Specifies a password with which to encrypt the database master key. password must meet the SQL Server PDW password policy.

Security note Security Note

Communication traffic between components inside the SQL Server PDW is not encrypted. High privileged users could potentially install network sniffing software to obtain the DMK password. Mitigate this risk by controlling physical access to the SQL Server PDW.

Remarks

The REGENERATE option re-creates the database master key and all the keys it protects. The keys are first decrypted with the old master key, and then encrypted with the new master key. This resource-intensive operation should be scheduled during a period of low demand, unless the master key has been compromised.

SQL Server PDW uses the AES encryption algorithm to protect the service master key (SMK) and the database master key (DMK). AES is a newer encryption algorithm than 3DES used in earlier versions.

When the FORCE option is used, key regeneration will continue even if the master key is unavailable or the server cannot decrypt all the encrypted private keys. Use the FORCE option only if the master key is irretrievable or if decryption fails. Information that is encrypted only by an irretrievable key will be lost.

The DROP ENCRYPTION BY SERVICE MASTER KEY option removes the encryption of the database master key by the service master key.

ADD ENCRYPTION BY SERVICE MASTER KEY causes a copy of the master key to be encrypted using the service master key and stored in both the current database and in master.

Permissions

Requires CONTROL SERVER permission.

Examples

The following example creates a new database master key for AdventureWorksPDW2012 and re-encrypts the keys below it in the encryption hierarchy.

Copy image Copy Code
USE master;
ALTER MASTER KEY REGENERATE WITH ENCRYPTION BY PASSWORD = 'dsjdkflJ435907NnmM#sX003';
GO

See Also

Reference

CREATE MASTER KEY (SQL Server PDW)
DROP MASTER KEY (SQL Server PDW)
OPEN MASTER KEY (SQL Server PDW)
CLOSE MASTER KEY (SQL Server PDW)
sys.databases (SQL Server PDW)


ALTER ROLE (SQL Server PDW)

[image]

ALTER ROLE (SQL Server PDW)
See Also Send Feedback

Changes the name of a database role.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER ROLE role_name WITH NAME = new_name

Arguments

role_name

Is the name of the role to be changed.

WITH NAME =new_name

Specifies the new name of the role. This name must not already exist in the database.

Remarks

Changing the name of a database role does not change ID number, owner, or permissions of the role.

Database roles are visible in the sys.database_role_members and sys.database_principals catalog views.

Permissions

Requires ALTER ANY ROLE permission on the database, or ALTER permission on the role, or membership in the db_securityadmin.

Examples

The following example changes the name of role buyers to purchasing.

Copy image Copy Code
USE AdventureWorks2008R2;
ALTER ROLE buyers WITH NAME = purchasing;
GO

See Also

Reference

CREATE ROLE (SQL Server PDW)
DROP ROLE (SQL Server PDW)
sp_addrolemember (SQL Server PDW)
sp_droprolemember (SQL Server PDW)
sys.database_principals (SQL Server PDW)
sys.database_permissions (SQL Server PDW)
sys.database_role_members (SQL Server PDW)


ALTER SERVER ROLE (SQL Server PDW)

[image]

ALTER SERVER ROLE (SQL Server PDW)
See Also Send Feedback

Adds and removes logins from a server role in SQL Server PDW. Use this statement to manage the capabilities of logins.

In this release, the built-in server roles are called resource classes. They are pre-configured to manage resource allocations for requests. For more information, see Workload Management (SQL Server PDW).

User-defined server roles are not available in this release of SQL Server PDW. To define a role, use user-defined database roles instead. For more information, see ALTER ROLE (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER SERVER ROLE  server_role_name  ADD MEMBER login;

ALTER SERVER ROLE  server_role_name  DROP MEMBER login;

Arguments

server_role_name

Specifies the name of the server role to be changed.

In this release, the only valid values for server_role_name are the resource classes MediumRC, LargeRC, and XLargeRC.

To understand the details of resource classes and how to use them to manage resources, see Workload Management (SQL Server PDW).

ADD MEMBER login

Specifies the login to add to the server role.

DROP MEMBER login

Specifies to the login to drop from the server role.

Permissions

Requires the CONTROL SERVER permission, or membership in one of the following server roles:

  • sysadmin

  • server_role_name. For example, a member of the XLargeRC server role can add a member to the XLargeRC server role.

A login can drop itself from any of the server roles.

General Remarks

SQL Server PDW uses resource classes for workload management. For more information, see Workload Management (SQL Server PDW).

The sa login cannot be added to the special roles for resource classes.

Examples

A. Basic Syntax

The following example adds the login Anna to the LargeRC server role.

Copy image Copy Code
ALTER SERVER ROLE LargeRC ADD MEMBER Anna;

B. Remove a login from a resource class.

The following example drops Anna’s membership in the LargeRC server role.

Copy image Copy Code
ALTER SERVER ROLE LargeRC DROP MEMBER Anna;

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Data Definition Statements (SQL Server PDW)
Common Metadata Query Examples (SQL Server PDW)
Workload Management (SQL Server PDW)


ALTER USER (SQL Server PDW)

[image]

ALTER USER (SQL Server PDW)
See Also Send Feedback

Renames a database user or changes its default schema in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

ALTER USER userName
     WITH <set_item> [ ,...n ]

<set_item> ::=
     NAME = newUserName
     | LOGIN = loginName
     | DEFAULT_SCHEMA = schema_name
[;]

Arguments

userName

Specifies the name by which the user is identified inside this database.

NAME =newUserName

Specifies the new name for this user. newUserName must not already occur in the current database.

LOGIN =loginName

Re-maps a user to another login by changing the user's Security Identifier (SID) to match the login's SID.

DEFAULT_SCHEMA = schema_name

Specifies the first schema that will be searched by the server when it resolves the names of objects for this database user.

Remarks

The WITH LOGIN clause enables the remapping of a user to a different login.

The name of the user will be automatically renamed to the login name if the following conditions are true.

  • No new name was specified.

  • The current name differs from the login name.

Otherwise, the user will not be renamed unless the caller additionally invokes the NAME clause.

The name of a user mapped to a SQL Server login cannot contain the backslash character (\).

Note Note

In SQL Server a default schema can be explicitly set to NULL for a database user that belongs to a Windows group. This is not possible in PDW. To work around this issue, either drop and recreate the user without specifying a default schema (preferred), or set the default schema of the user to a schema that does not exist.

Permissions

To change the name of a user requires the ALTER ANY USER permission.

To change the user name, target login, or SID of a user having CONTROL permission on the database requires the CONTROL permission on the database.

To change the default schema requires ALTER permission on the user. All users can change their own default schema.

Locking

Takes an exclusive lock on the SCHEMARESOLUTION object. If the default schema clause is specified, takes an exclusive lock on the DATABASE object. If the default schema clause is not specified, takes a shared lock on the DATABASE object.

Examples

A. Changing the name of a database user

The following example changes the name of the database user Mary5 to Mary51.

Copy image Copy Code
USE AdventureWorks2008R2;
ALTER USER Mary5 WITH NAME = Mary51;
GO

B. Changing the default schema

The following example changes the default schema of the database user Contoso\Mary to Sales.

Copy image Copy Code
ALTER USER [Contoso\Mary] WITH DEFAULT_SCHEMA = Sales;

See Also

Reference

CREATE USER (SQL Server PDW)
DROP USER (SQL Server PDW)
sys.database_principals (SQL Server PDW)


BACKUP CERTIFICATE (SQL Server PDW)

[image]

BACKUP CERTIFICATE (SQL Server PDW)
See Also Send Feedback

Exports a certificate to a file. Can only be executed in the master database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

BACKUP CERTIFICATE certname TO FILE = 'path_to_file'
      WITH PRIVATE KEY
      (
        FILE = 'path_to_private_key_file' ,
        ENCRYPTION BY PASSWORD = 'encryption_password'
      )

Arguments

path_to_file

Specifies the complete path, including file name, of the file in which the certificate is to be saved. This must be a UNC path to a network location.

path_to_private_key_file

Specifies the complete path, including file name, of the file in which the private key is to be saved. This must be a UNC path to a network location.

encryption_password

Is the password that is used to encrypt the private key before writing the key to the backup file. The password is subject to complexity checks.

Remarks

When you back up the private key to a file, encryption is required. The password used to protect the backed up certificate is not the same password that is used to encrypt the private key of the certificate.

To restore a backed up certificate, use the CREATE CERTIFICATE (SQL Server PDW) statement.

Permissions

Requires CONTROL SERVER permission.

Examples

A. Exporting a certificate and a private key

In the following example, the private key of the certificate that is backed up will be encrypted with the password 997jkhUbhk$w4ez0876hKHJH5gh.

Copy image Copy Code
BACKUP CERTIFICATE sales05 TO FILE = '\\ServerA7\storedcerts\sales05cert'
    WITH PRIVATE KEY ( FILE = '\\ServerA7\storedkeys\sales05key' ,
    ENCRYPTION BY PASSWORD = '997jkhUbhk$w4ez0876hKHJH5gh' );
GO

See Also

Reference

CREATE CERTIFICATE (SQL Server PDW)
ALTER CERTIFICATE (SQL Server PDW)
DROP CERTIFICATE (SQL Server PDW)
sys.certificates (SQL Server PDW)


CLOSE MASTER KEY (SQL Server PDW)

[image]

CLOSE MASTER KEY (SQL Server PDW)
See Also Send Feedback

Closes the master key of the current database. Must be executed in the master database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CLOSE MASTER KEY

Arguments

Takes no arguments.

Remarks

This statement reverses the operation performed by OPEN MASTER KEY. CLOSE MASTER KEY only succeeds when the database master key was opened in the current session by using the OPEN MASTER KEY statement.

Permissions

Requires CONTROL SERVER permission to open the master key. The master key can be closed in the same session as it was opened without additional permissions.

Examples

Copy image Copy Code
USE master;
OPEN MASTER KEY DECRYPTION BY PASSWORD = '43987hkhj4325tsku7';
GO
CLOSE MASTER KEY;
GO

See Also

Reference

OPEN MASTER KEY (SQL Server PDW)
CREATE MASTER KEY (SQL Server PDW)
ALTER MASTER KEY (SQL Server PDW)
DROP MASTER KEY (SQL Server PDW)
sys.databases (SQL Server PDW)


CREATE CERTIFICATE (SQL Server PDW)

[image]

CREATE CERTIFICATE (SQL Server PDW)
See Also Send Feedback

Adds a certificate to a database in SQL Server PDW. Can only be executed in the master database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE CERTIFICATE certificate_name
    { <generate_new_keys> | FROM <existing_keys> }
    [ ; ]

<generate_new_keys> ::=
    WITH SUBJECT = 'certificate_subject_name'
    [ , <date_options> [ ,...n ] ]

<existing_keys> ::=
    {
      FILE = 'path_to_file'
      WITH PRIVATE KEY
         (
           FILE = 'path_to_private_key'
           , DECRYPTION BY PASSWORD = 'password'
         )
    }

<date_options> ::=
    START_DATE = 'datetime' | EXPIRY_DATE = 'datetime'

Arguments

certificate_name

Is the name by which the certificate will be known in the database.

FILE ='path_to_file'

Specifies the complete path, including file name, to a DER-encoded file that contains the certificate. path_to_file must be a UNC path to a network location. The file will be accessed in the security context of a credential created by using the sp_pdw_add_network_credentials statement. This credential account must have the required file-system permissions.

WITH PRIVATE KEY

Specifies that the private key of the certificate is loaded into SQL Server PDW. This clause is only valid when the certificate is being created from a file.

FILE ='path_to_private_key'

Specifies the complete path, including file name, to the private key. path_to_private_key must be a UNC path to a network location. The file will be accessed in the security context of a credential created by using the sp_pdw_add_network_credentials statement. This credential account must have the necessary file-system permissions.

DECRYPTION BY PASSWORD ='key_password'

Specifies the password required to decrypt a private key that is retrieved from a file.

SUBJECT ='certificate_subject_name'

The term subject refers to a field in the metadata of the certificate as defined in the X.509 standard. The subject can be up to 128 characters long. Subjects that exceed 128 characters will be truncated when they are stored in the catalog, but the binary large object (BLOB) that contains the certificate will retain the full subject name.

START_DATE ='datetime'

Is the date on which the certificate becomes valid. If not specified, START_DATE will be set equal to the current date. START_DATE is in UTC time and can be specified in any format that can be converted to a date and time. Though a valid parameter, start date is ignored for SQL Server PDW TDE.

EXPIRY_DATE ='datetime'

Is the date on which the certificate expires. If not specified, EXPIRY_DATE will be set to a date one year after START_DATE. EXPIRY_DATE is in UTC time and can be specified in any format that can be converted to a date and time. Though a valid parameter, start date is ignored for SQL Server PDW TDE.

Remarks

Private keys generated by SQL Server are 1024 bits long. Private keys imported from an external source have a minimum length of 384 bits and a maximum length of 4,096 bits. The length of an imported private key must be an integer multiple of 64 bits.

The private key must correspond to the public key specified by certificate_name.

Security note Security Note

Communication traffic between components inside the SQL Server PDW is not encrypted. High privileged users could potentially install network sniffing software to obtain the certificate private and public keys. Mitigate this risk by controlling physical access to the SQL Server PDW.

Permissions

Requires CONTROL SERVER permission.

Examples

A. Creating a self-signed certificate

The following example creates a certificate called Shipping04.

Copy image Copy Code
USE master;
GO
CREATE CERTIFICATE Shipping04
   WITH SUBJECT = 'Sammamish Shipping Records';
GO

B. Creating a certificate from a file

The following example creates a certificate in the database, loading the key pair from files.

Copy image Copy Code
USE master;
GO
CREATE CERTIFICATE Shipping11
    FROM FILE = '\\Server7A\Shipping\Certs\Shipping11.cer'
    WITH PRIVATE KEY (FILE = '\\Server7A\Shipping\Certs\Shipping11.pvk',
    DECRYPTION BY PASSWORD = 'sldkflk34et6gs%53#v00');
GO

See Also

Reference

ALTER CERTIFICATE (SQL Server PDW)
DROP CERTIFICATE (SQL Server PDW)
BACKUP CERTIFICATE (SQL Server PDW)
sys.certificates (SQL Server PDW)


CREATE DATABASE ENCRYPTION KEY (SQL Server PDW)

[image]

CREATE DATABASE ENCRYPTION KEY (SQL Server PDW)
See Also Send Feedback

Creates an encryption key that is used for transparently encrypting a database. For more information about transparent database encryption, see Transparent Data Encryption (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE DATABASE ENCRYPTION KEY
       WITH ALGORITHM = { AES_128 | AES_192 | AES_256 | TRIPLE_DES_3KEY }
   ENCRYPTION BY SERVER CERTIFICATE Encryptor_Name
[ ; ]

Arguments

WITH ALGORITHM = { AES_128 | AES_192 | AES_256 | TRIPLE_DES_3KEY }

Specifies the encryption algorithm that is used for the encryption key.

ENCRYPTION BY SERVER CERTIFICATE Encryptor_Name

Specifies the name of the encryptor used to encrypt the database encryption key.

Remarks

A database encryption key is required before a database can be encrypted by using TDE. The certificate that is used to encrypt the database encryption key must be located in the master system database.

Database encryption statements are allowed only on user databases.

The database encryption key cannot be exported from the database. It is available only to the system, to users who have debugging permissions on the server, and to users who have access to the certificates that encrypt and decrypt the database encryption key.

The database encryption key does not have to be regenerated when a database owner (dbo) is changed.

Permissions

Requires CONTROL SERVER permission.

Examples

For additional examples using TDE, see Transparent Data Encryption (SQL Server PDW).

The following example creates a database encryption key by using the AES_256 algorithm, and protects the private key with a certificate named MyServerCert.

Copy image Copy Code
USE AdventureWorksPDW2012;
GO
CREATE DATABASE ENCRYPTION KEY
WITH ALGORITHM = AES_256
ENCRYPTION BY SERVER CERTIFICATE MyServerCert;
GO

See Also

Reference

ALTER DATABASE ENCRYPTION KEY (SQL Server PDW)
DROP DATABASE ENCRYPTION KEY (SQL Server PDW)
sys.dm_pdw_nodes_database_encryption_keys (SQL Server PDW)
sp_pdw_database_encryption_regenerate_system_keys (SQL Server PDW)

Concepts

Transparent Data Encryption (SQL Server PDW)


CREATE LOGIN (SQL Server PDW)

[image]

CREATE LOGIN (SQL Server PDW)
See Also Send Feedback

Creates a login account to access a SQL Server PDW appliance. Use this statement for managing appliance security.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE LOGIN loginName { WITH <option_list1> | FROM WINDOWS }

<option_list1> ::=
    PASSWORD = { 'password' } [ MUST_CHANGE ]
    [ , <option_list2> [ ,... ] ]

<option_list2> ::=
      CHECK_EXPIRATION = { ON | OFF}
    | CHECK_POLICY = { ON | OFF}

Arguments

login_name

Specifies the name of the login that is created. SQL Server PDW supports two types of logins: SQL Server logins and Windows logins. When you are creating logins that are mapped from a Windows domain account, you must use the pre-Windows 2000 user logon name in the format [appliance_domain\<login_name>]. You cannot use a UPN in the format login_name@DomainName. SQL Server authentication logins are type sysname and must conform to the rules for Object Naming Rules (SQL Server PDW) and cannot contain a '\'. Windows logins can contain a '\'.

password

Applies to SQL Server logins only. The password to use for the initial login. Passwords are case-sensitive. Passwords should always be at least 8 characters long, and cannot exceed 128 characters. Passwords can include a-z, A-Z, 0-9, and most non-alphanumeric characters. Passwords cannot contain single quotes, or the login_name. Additionally, passwords must contain at least three of the following:

  • An uppercase character (A-Z).

  • A lowercase character (a-z).

  • A digit (0-9).

  • One of the non-alphanumeric characters, such as a space, _, @, *, ^, %, !, $, #, or &.

Caution note Caution

The characters ^ % & are permitted in passwords, however PowerShell regards these as special characters. If these characters are used in passwords for the system administrator or SQL Server sa accounts (the AdminPassword and PdwSAPassword parameters during setup) then setup, including INSTALL, UPGRADE, REPLACENODE, and PATCHING, will fail. To ensure a successful upgrade when current passwords contain unsupported characters, change these passwords so that they do not contain such characters before running upgrade. After upgrade completes, you can set these passwords back to their original values.

MUST_CHANGE

Applies to SQL Server logins only. If this option is included, SQL Server prompts the user for a new password the first time the new login is used.

CHECK_EXPIRATION = { ON | OFF }

Applies to SQL Server logins only. Specifies whether password expiration policy should be enforced on this login. The default value is OFF.

Applies to SQL Server logins only. CHECK_POLICY = { ON | OFF }

Specifies that the Windows password policies of the SQL Server PDW system on which SQL Server is running should be enforced on this login. The default value is ON.

WINDOWS

Specifies that the login be mapped to a Windows login.

Permissions

Requires ALTER ANY LOGIN permission.

General Remarks

Creating a login automatically enables the new login and grants the login CONNECT SQL permission to the SQL Server PDW. To connect to a database, create a database user for the login. For more information, see CREATE USER (SQL Server PDW). For more information about granting and denying permissions to logins and users, see Permissions: GRANT, DENY, REVOKE (SQL Server PDW).

If MUST_CHANGE is specified, CHECK_EXPIRATION and CHECK_POLICY must be set to ON. Otherwise, the statement will fail.

A combination of CHECK_POLICY = OFF and CHECK_EXPIRATION = ON is not supported.

When CHECK_POLICY is set to OFF, the lockout_time is reset and CHECK_EXPIRATION is set to OFF.

Default Windows Policies

The following policies are inherited from the Windows domain.

  • Password Policies

    • Enforced password history: 24 passwords remembered

    • Maximum password age: 42 days

    • Minimum password age: 1 day

    • Minimum password length: 7 characters

      Security note Security Note

      Passwords are encrypted using blocks of 7 characters (plus a check character). Passwords should always be at least 8 characters long so that at least two encryption blocks are required, greatly increasing the complexity of the encryption.

    • Password must meet complexity requirements: Enabled

    • Store password using reversible encryption: Disabled

  • Account Lockout Policies

    • Account lockout duration: 30 minutes

    • Account lockout threshold: 10 invalid login attempts

    • Reset account lockout counter after: 30 minutes

  • Password complexity policies are listed at Password Policy.

Note Note

Changing the Analytics Platform System appliance domain account lockout and password policies is not supported.

Examples

A. Creating a SQL Server authentication login with a password

The following example creates the login Mary7 with password A2c3456.

Transact-SQL Copy image Copy Code
CREATE LOGIN Mary7 WITH PASSWORD = 'A2c3456$#' ;

B. Using Options

The following example creates the login Mary8 with password some of the optional arguments.

Copy image Copy Code
CREATE LOGIN Mary8 WITH PASSWORD = 'A2c3456$#' MUST_CHANGE,
CHECK_EXPIRATION = ON,
CHECK_POLICY = ON;

C. Creating a login from a Windows domain account

The following example creates a login from a Windows domain account named Mary in the Contoso domain.

Copy image Copy Code
CREATE LOGIN [Contoso\Mary] FROM WINDOWS;
GO

See Also

Reference

DROP LOGIN (SQL Server PDW)
ALTER LOGIN (SQL Server PDW)


CREATE MASTER KEY (SQL Server PDW)

[image]

CREATE MASTER KEY (SQL Server PDW)
See Also Send Feedback

Creates a database master key. This statement must be executed in the master database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password'

Arguments

PASSWORD ='password'

Is the password that is used to encrypt the master key in the database. password must meet the SQL Server PDW password policy.

Security note Security Note

Communication traffic between components inside the SQL Server PDW is not encrypted. High privileged users could potentially install network sniffing software to obtain the DMK password. Mitigate this risk by controlling physical access to the SQL Server PDW.

Remarks

The database master key is a symmetric key used to protect the database encryption key during transparent data encryption. For more information about TDE, see Transparent Data Encryption (SQL Server PDW). When it is created, the master key is encrypted by using the AES_256 algorithm and a user-supplied password. To enable the automatic decryption of the master key, a copy of the key is encrypted by using the service master key and stored in in the master database. This default can be changed by using the DROP ENCRYPTION BY SERVICE MASTER KEY option of ALTER MASTER KEY (SQL Server PDW). A master key that is not encrypted by the service master key must be opened by using the OPEN MASTER KEY (SQL Server PDW) statement and a password.

The is_master_key_encrypted_by_server column of the sys.databases catalog view in master indicates whether the database master key is encrypted by the service master key.

Information about the database master key is visible in the sys.symmetric_keys catalog view.

The service master key and database master keys are protected by using the AES-256 algorithm.

Permissions

Requires CONTROL SERVER.

Examples

The following example creates a master key. The key is encrypted using the password 23987hxJ#KL95234nl0zBe.

Copy image Copy Code
USE master;
CREATE MASTER KEY ENCRYPTION BY PASSWORD = '23987hxJ#KL95234nl0zBe';
GO

See Also

Reference

ALTER MASTER KEY (SQL Server PDW)
DROP MASTER KEY (SQL Server PDW)
OPEN MASTER KEY (SQL Server PDW)
CLOSE MASTER KEY (SQL Server PDW)
sys.databases (SQL Server PDW)


CREATE ROLE (SQL Server PDW)

[image]

CREATE ROLE (SQL Server PDW)
See Also Send Feedback

Creates a new database role in the SQL Server PDW current database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE ROLE role_name [ AUTHORIZATION owner_name ]

Arguments

role_name

Is the name of the role to be created.

AUTHORIZATION owner_name

Is the database user or role that is to own the new role. If no user is specified, the role will be owned by the user that executes CREATE ROLE.

Remarks

Roles are database-level securables. After you create a role, configure the database-level permissions of the role by using GRANT, DENY, and REVOKE. To add members to a database role, use the sp_addrolemember stored procedure. For more information, see sys.database_principals (SQL Server PDW).

Database roles are visible in the sys.database_role_members and sys.database_principals catalog views.

Permissions

Requires CREATE ROLE permission on the database or membership in the db_securityadmin fixed database role. When you use the AUTHORIZATION option, the following permissions are also required:

  • To assign ownership of a role to another user, requires IMPERSONATE permission on that user.

  • To assign ownership of a role to another role, requires membership in the recipient role or ALTER permission on that role.

  • To assign ownership of a role to an application role, requires ALTER permission on the application role.

Examples

A. Creating a database role that is owned by a database user

The following example creates the database role buyers that is owned by user BenMiller.

Copy image Copy Code
USE AdventureWorks2008R2;
CREATE ROLE buyers AUTHORIZATION BenMiller;
GO

B. Creating a database role that is owned by a fixed database role

The following example creates the database role auditors that is owned the db_securityadmin fixed database role.

Copy image Copy Code
USE AdventureWorks2008R2;
CREATE ROLE auditors AUTHORIZATION db_securityadmin;
GO

See Also

Reference

ALTER ROLE (SQL Server PDW)
DROP ROLE (SQL Server PDW)
sp_addrolemember (SQL Server PDW)
sp_droprolemember (SQL Server PDW)
sys.database_role_members (SQL Server PDW)
sys.database_principals (SQL Server PDW)


CREATE SERVER ROLE (SQL Server PDW)

[image]

CREATE SERVER ROLE (SQL Server PDW)
See Also Send Feedback

User-defined server roles are not available in this release of SQL Server PDW. Use user-defined database roles instead. For more information, see CREATE ROLE (SQL Server PDW).

Syntax

  See Also
Reference
ALTER SERVER ROLE (SQL Server PDW)
Concepts
Common Metadata Query Examples (SQL Server PDW)

            Send

      feedback

            on this topic to Microsoft.


CREATE USER (SQL Server PDW)

[image]

CREATE USER (SQL Server PDW)
See Also Send Feedback

Adds a user to the current SQL Server PDW database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

CREATE USER user_name
    [ { { FOR | FROM }
      {
        LOGIN login_name
      }
      | WITHOUT LOGIN
    ]
    [ WITH DEFAULT_SCHEMA = schema_name ]
[;]

Arguments

user_name

Specifies the name by which the user is identified inside this database. user_name can be up to 128 characters long. When creating a user based on a Windows principal, the Windows principal name becomes the user name unless another user name is specified.

LOGIN login_name

Specifies the login for which the database user is being created. login_name must be a valid login in the server. Can be a login based on a Windows principal (user or group), or a login created by using SQL Server authentication. When this login enters the database it will acquire the name and ID of the database user that is being created. When creating a login mapped from a Windows principal, use the format [<domainName>\<loginName>].

WITHOUT LOGIN

Specifies that the user should not be mapped to an existing login.

WITH DEFAULT_SCHEMA = schema_name

Specifies the first schema that will be searched by the server when it resolves the names of objects for this database user.

Remarks

If FOR LOGIN is omitted, the new database user will be mapped to the SQL Server PDW login with the same name.

The default schema will be the first schema that will be searched by the server when it resolves the names of objects for this database user. Unless otherwise specified, the default schema will be the owner of objects created by this database user.

If the user has a default schema, that default schema will used. If the user does not have a default schema, but the user is a member of a group that has a default schema, the default schema of the group will be used. If the user does not have a default schema, and is a member of more than one group, the default schema for the user will be that of the Windows group with the lowest principal_id and an explicitly set default schema. (It is not possible to explicitly select one of the available default schemas as the preferred schema.) If no default schema can be determined for a user, the dbo schema will be used.

DEFAULT_SCHEMA can be set before the schema that it points to is created.

The value of DEFAULT_SCHEMA is ignored if the user is a member of the sysadmin fixed server role. All members of the sysadmin fixed server role have a default schema of dbo.

The WITHOUT LOGIN clause creates a user that is not mapped to a SQL Server PDW login. It can connect to other databases as guest.

The names of users that are mapped to SQL Server PDW logins cannot contain the backslash character (\).

CREATE USER cannot be used to create a guest user because the guest user already exists inside every database. You can enable the guest user by granting it CONNECT permission, as shown:

Copy image Copy Code
GRANT CONNECT TO guest;
GO

Information about database users is visible in the sys.database_principals (SQL Server PDW) catalog view.

Permissions

Requires ALTER ANY USER permission on the database.

Locking

Takes an exclusive lock on the SCHEMARESOLUTION object. If the default schema clause is specified, takes an exclusive lock on the DATABASE object. If the default schema clause is not specified, takes a shared lock on the DATABASE object.

Examples

A. Creating a database user

The following example first creates a server login named AbolrousHazem with a password, and then creates a corresponding database user AbolrousHazem in AdventureWorks2008R2.

Copy image Copy Code
CREATE LOGIN AbolrousHazem
    WITH PASSWORD = '340$Uuxwp7Mcxo7Khy';
USE AdventureWorks2008R2;
CREATE USER AbolrousHazem FOR LOGIN AbolrousHazem;
GO

See Also

Reference

ALTER USER (SQL Server PDW)
DROP USER (SQL Server PDW)
CREATE LOGIN (SQL Server PDW)


DROP CERTIFICATE (SQL Server PDW)

[image]

DROP CERTIFICATE (SQL Server PDW)
See Also Send Feedback

Removes a certificate from a SQL Server PDW database. Can only be executed in the master database.

Important note Important

The encrypting certificate should be retained even if TDE is no longer enabled on the database. Even though the database is not encrypted, parts of the transaction log may still remain protected, and the certificate may be needed for some operations until the full backup of the database is performed.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP CERTIFICATE certificate_name

Arguments

certificate_name

Is the unique name by which the certificate is known in the database.

Remarks

Certificates can only be dropped if no entities are associated with them.

Permissions

Requires CONTROL SERVER permission.

Examples

The following example drops the certificate Shipping04.

Copy image Copy Code
USE master;
DROP CERTIFICATE Shipping04;

See Also

Reference

CREATE CERTIFICATE (SQL Server PDW)
ALTER CERTIFICATE (SQL Server PDW)
BACKUP CERTIFICATE (SQL Server PDW)
sys.certificates (SQL Server PDW)


DROP DATABASE ENCRYPTION KEY (SQL Server PDW)

[image]

DROP DATABASE ENCRYPTION KEY (SQL Server PDW)
See Also Send Feedback

Drops a SQL Server PDW database encryption key that is used in transparent database encryption. For more information about transparent database encryption, see Transparent Data Encryption (SQL Server PDW).

Important note Important

The backup of the certificate that was protecting the database encryption key should be retained even if the encryption is no longer enabled on a database. Even though the database is not encrypted anymore, parts of the transaction log may still remain protected, and the certificate may be needed for some operations until the full backup of the database is performed.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP DATABASE ENCRYPTION KEY

Remarks

If the database is encrypted, you must first remove encryption from the database by using the ALTER DATABASE statement. Wait for decryption to complete before removing the database encryption key. For more information about the ALTER DATABASE statement, see ALTER DATABASE (SQL Server PDW). To view the state of the database, use the sys.dm_pdw_nodes_database_encryption_keys (SQL Server PDW) dynamic management view.

Permissions

Requires CONTROL permission on the database.

Examples

The following example removes the TDE encryption and then drops the database encryption key.

Copy image Copy Code
ALTER DATABASE AdventureWorksPDW2012
    SET ENCRYPTION OFF;
GO
/* Wait for decryption operation to complete, look for a
value of  1 in the query below. */
WITH dek_encryption_state AS
(
    SELECT ISNULL(db_map.database_id, dek.database_id) AS database_id, encryption_state
    FROM sys.dm_pdw_nodes_database_encryption_keys AS dek
        INNER JOIN sys.pdw_nodes_pdw_physical_databases AS node_db_map
           ON dek.database_id = node_db_map.database_id AND dek.pdw_node_id = node_db_map.pdw_node_id
        LEFT JOIN sys.pdw_database_mappings AS db_map
            ON node_db_map .physical_name = db_map.physical_name
        INNER JOIN sys.dm_pdw_nodes AS nodes
            ON nodes.pdw_node_id = dek.pdw_node_id
    WHERE dek.encryptor_thumbprint <> 0x
)
SELECT TOP 1 encryption_state
       FROM  dek_encryption_state
       WHERE dek_encryption_state.database_id = DB_ID('AdventureWorksPDW2012 ')
       ORDER BY (CASE encryption_state WHEN 3 THEN -1 ELSE encryption_state END) DESC;
GO
USE AdventureWorksPDW2012;
GO
DROP DATABASE ENCRYPTION KEY;
GO

See Also

Reference

CREATE DATABASE ENCRYPTION KEY (SQL Server PDW)
ALTER DATABASE ENCRYPTION KEY (SQL Server PDW)
sys.dm_pdw_nodes_database_encryption_keys (SQL Server PDW)
sp_pdw_database_encryption_regenerate_system_keys (SQL Server PDW)

Concepts

Transparent Data Encryption (SQL Server PDW)


DROP LOGIN (SQL Server PDW)

[image]

DROP LOGIN (SQL Server PDW)
See Also Send Feedback

Drops or deletes a login in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP LOGIN login_name
[;]

Arguments

login_name

The name of the login to drop or delete.

Permissions

Requires ALTER ANY LOGIN permission. A login cannot be dropped while it is logged in.

Limitations and Restrictions

To drop a login that owns one or more securables, you must first delete the securables or use the ALTER AUTHORIZATION (SQL Server PDW) statement to change the securable owner to a different login.

Examples

A. Dropping a login

The following example removes database login login7 from the appliance.

Copy image Copy Code
DROP LOGIN login7;

See Also

Reference

CREATE LOGIN (SQL Server PDW)
ALTER LOGIN (SQL Server PDW)


DROP MASTER KEY (SQL Server PDW)

[image]

DROP MASTER KEY (SQL Server PDW)
See Also Send Feedback

Removes the master key. Must be executed in the master database. SQL Server PDW uses the master key to protect a certificate, which is used by transparent data encryption. For more information, see Transparent Data Encryption (SQL Server PDW).

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP MASTER KEY

Arguments

This statement takes no arguments.

Remarks

The drop will fail if a certificate is protected by the master key.

Permissions

Requires CONTROL SERVER permission.

Examples

The following example removes the master key.

Copy image Copy Code
USE master;
DROP MASTER KEY;
GO

See Also

Reference

CREATE MASTER KEY (SQL Server PDW)
ALTER MASTER KEY (SQL Server PDW)
OPEN MASTER KEY (SQL Server PDW)
CLOSE MASTER KEY (SQL Server PDW)
sys.databases (SQL Server PDW)


DROP ROLE (SQL Server PDW)

[image]

DROP ROLE (SQL Server PDW)
See Also Send Feedback

Removes a role from the database in SQL Server PDW.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP ROLE role_name

Arguments

role_name

Specifies the role to be dropped from the database.

Remarks

Roles that own securables cannot be dropped from the database. To drop a database role that owns securables, you must first transfer ownership of those securables or drop them from the database. Roles that have members cannot be dropped from the database. To drop a role that has members, you must first remove members of the role.

You cannot use DROP ROLE to drop a fixed database role.

Information about role membership can be viewed in the sys.database_role_members catalog view.

Permissions

Requires ALTER ANY ROLE permission on the database, or CONTOL permission on the role, or membership in the db_securityadmin.

Examples

The following example drops the database role purchasing from AdventureWorks2008R2.

Copy image Copy Code
USE AdventureWorks2008R2;
DROP ROLE purchasing;
GO

See Also

Reference

CREATE ROLE (SQL Server PDW)
ALTER ROLE (SQL Server PDW)
sp_addrolemember (SQL Server PDW)
sp_droprolemember (SQL Server PDW)
sys.database_principals (SQL Server PDW)
sys.database_permissions (SQL Server PDW)
sys.database_role_members (SQL Server PDW)


DROP SERVER ROLE (SQL Server PDW)

[image]

DROP SERVER ROLE (SQL Server PDW)
See Also Send Feedback

User-defined server roles are not available in this release of SQL Server PDW. Use user-defined database roles instead. For more information, see DROP ROLE (SQL Server PDW).

Syntax

  See Also
Concepts
Common Metadata Query Examples (SQL Server PDW)

            Send

      feedback

            on this topic to Microsoft.


DROP USER (SQL Server PDW)

[image]

DROP USER (SQL Server PDW)
See Also Send Feedback

Removes a user from the current SQL Server PDW database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

DROP USER user_name

Arguments

user_name

Specifies the name by which the user is identified inside this database.

Remarks

Users that own securables cannot be dropped from the database. Before dropping a database user that owns securables, you must first drop or transfer ownership of those securables.

The guest user cannot be dropped, but guest user can be disabled by revoking its CONNECT permission by executing REVOKE CONNECT FROM GUEST within any database other than master or tempdb.

Permissions

Requires ALTER ANY USER permission on the database.

Locking

Takes an exclusive lock on the SCHEMARESOLUTION and DATABASE objects.

Examples

The following example removes database user AbolrousHazem from the AdventureWorks2008R2 database.

Copy image Copy Code
USE AdventureWorks2008R2;
DROP USER AbolrousHazem;
GO

See Also

Reference

ALTER USER (SQL Server PDW)
CREATE USER (SQL Server PDW)
sys.database_principals (SQL Server PDW)


OPEN MASTER KEY (SQL Server PDW)

[image]

OPEN MASTER KEY (SQL Server PDW)
See Also Send Feedback

Opens the database master key in the master database.

Topic link icon Syntax Conventions (SQL Server PDW)

Syntax

OPEN MASTER KEY DECRYPTION BY PASSWORD = 'password' [ ; ]

Arguments

'password'

The password with which the database master key was encrypted.

Security note Security Note

Communication traffic between components inside the SQL Server PDW is not encrypted. High privileged users could potentially install network sniffing software to obtain the DMK password. Mitigate this risk by controlling physical access to the SQL Server PDW.

Remarks

If the database master key was encrypted with the service master key, it will be automatically opened when it is needed for decryption or encryption. In this case, it is not necessary to use the OPEN MASTER KEY statement.

When a database is first attached or restored to a new instance of SQL Server PDW, a copy of the database master key (encrypted by the service master key) is not yet stored in the server. You must use the OPEN MASTER KEY statement to decrypt the database master key (DMK). Once the DMK has been decrypted, you have the option of enabling automatic decryption in the future by using the ALTER MASTER KEY REGENERATE statement to provision the server with a copy of the DMK encrypted with the service master key (SMK).

You can exclude the database master key from automatic key management by using the ALTER MASTER KEY statement with the DROP ENCRYPTION BY SERVICE MASTER KEY option. Afterward, you must explicitly open the database master key with a password.

If a transaction in which the database master key was explicitly opened is rolled back, the key will remain open.

Permissions

Requires CONTROL SERVER permission.

Examples

The following example opens the database master, which has been encrypted with a password.

Copy image Copy Code
USE master;
OPEN MASTER KEY DECRYPTION BY PASSWORD = '43987hkhj4325tsku7';
GO
CLOSE MASTER KEY;
GO

See Also

Reference

CREATE MASTER KEY (SQL Server PDW)
ALTER MASTER KEY (SQL Server PDW)
DROP MASTER KEY (SQL Server PDW)
CLOSE MASTER KEY (SQL Server PDW)
sys.databases (SQL Server PDW)


System Views (SQL Server PDW)

[image]

System Views (SQL Server PDW)
See Also Send Feedback

This topic explains basics about the system views in SQL Server PDW and points to the individual system view topics, and examples of common metadata queries. System views includes dynamic management views (DMVs), catalog views, and information schema views. Use system views to view the current state of the system, as well as values, objects, and settings.

Contents

System View Basics

System View Topics

  • Common Metadata Query Examples (SQL Server PDW)

  • External Operations Catalog Views (SQL Server PDW)

  • SQL Server PDW Catalog Views (SQL Server PDW)

  • SQL Server PDW Dynamic Management Views (SQL Server PDW)

  • SQL Server Catalog Views (SQL Server PDW)

  • SQL Server Dynamic Management Views (SQL Server PDW)

  • SQL Server Information_Schema Views (SQL Server PDW)

System View Basics

System views describe the metadata, the system catalog, and the dynamic processes for SQL Server PDW.

Key Terms

There are three types of views within system views: dynamic management views (DMVs), catalog views, and information schema views.

Dynamic Management Views (DMVs)

Dynamic Management Views (DMVs) show information about dynamic processes, such as the queries in progress and memory usage on each appliance node.

Catalog Views

Catalog views show information about metadata and static elements of SQL Server PDW, such as table and column names and principals.

Information Schema Views

Information schema view show metadata for the data objects stored in a database. The information schema views are defined in a special schema named INFORMATION_SCHEMA. This schema is contained in each database.

Understanding System Views in SQL Server PDW

In SQL Server PDW you will find the following views:

SQL Server PDW Views

These views describe the metadata, system state, and dynamic processes that are specific to the MPP SQL Server PDW system.

In SQL Server PDW:

  • The name of each SQL Server PDW catalog view begins with sys.pdw_.

  • The name of each SQL Server PDW DMV begins with sys.dm_pdw_

SQL Server Views

These views are provided by SQL Server running on the Compute nodes. For each SQL Server view, SQL Server PDW combines the views from each of the Compute nodes into one view.

In SQL Server PDW:

  • The name of each combined SQL Server DMV begins with ‘sys.dm_pdw_nodes’.

  • The name of each combined SQL Server catalog view is the same as the SQL Server name.

[TOP]

See Also


Common Metadata Query Examples (SQL Server PDW)

[image]

Common Metadata Query Examples (SQL Server PDW)
See Also Send Feedback

This topic provides examples of common metadata queries for SQL Server PDW.

Contents

  • Distributed and Replicated Tables

  • Partitioned Tables

  • Collations

  • Appliance Health and State

  • Security Information

  • Workload Management

Joining Tables for User-table Metadata

The following graphic shows some of the system tables and views that present metadata about your user-tables and indexes. The graphic can help you determine which tables and views you should join together in your queries. For a printable version of this graphic, see System Table chart on the Yammer site.

APS system tables for metadata

Distributed and Replicated Table Queries

These queries show how to find distribution-related information. They apply to the AdventureWorksPDW2012 database.

A. List all replicated tables

Copy image Copy Code
/* List all of the replicated tables */
USE
    AdventureWorksPDW2012
  ;
SELECT a.name AS [Table Name],
b.distribution_policy_desc AS [Distribution Type]
FROM sys.tables AS a
JOIN sys.pdw_table_distribution_properties AS b
    ON a.object_id = b.object_id
WHERE b.distribution_policy = 3;

B. List all distributed tables

Copy image Copy Code
/* List all of the distributed tables */
USE
    AdventureWorksPDW2012
  ;
SELECT a.name AS [Table Name],
b.distribution_policy_desc AS [Distribution Type]
FROM sys.tables AS a
JOIN sys.pdw_table_distribution_properties AS b
    ON a.object_id = b.object_id
WHERE b.distribution_policy = 2;

C. List the distribution type for all tables

Copy image Copy Code
/* List all the tables in
    AdventureWorksPDW2012
   and show whether they are replicated or HASH distributed */
USE
    AdventureWorksPDW2012
  ;
SELECT a.name AS [Table Name],
b.distribution_policy_desc AS [Distribution Type]
FROM sys.tables AS a
JOIN sys.pdw_table_distribution_properties AS b
    ON a.object_id = b.object_id;

D. List the distribution column for one table

Copy image Copy Code
/* List the distribution column for the table DimProduct
   Distribution column is ProductKey */
USE
    AdventureWorksPDW2012
  ;
SELECT a.name AS [Table Name], c.name AS [Column Name],
d.distribution_ordinal AS [Distributed]
FROM sys.pdw_column_distribution_properties AS d
INNER JOIN sys.columns AS c
    ON c.object_id = d.object_id
INNER JOIN sys.tables AS a
    ON a.object_id = d.object_id
WHERE a.name = 'DimProduct'
    AND d.distribution_ordinal = 1
    AND c.column_id = d.column_id;

E. List the distribution column for all distributed tables

Copy image Copy Code
/* List all of the distributed tables and their distribution columns */
USE
    AdventureWorksPDW2012
  ;
SELECT a.name AS [Table Name], c.name AS [Distribution Column],
b.name AS [Data Type], d.distribution_ordinal AS [Distributed]
FROM sys.pdw_column_distribution_properties AS d
INNER JOIN sys.columns AS c
    ON c.object_id = d.object_id
INNER JOIN sys.tables AS a
    ON a.object_id = d.object_id
LEFT OUTER JOIN sys.types AS b
    ON c.user_type_id = b.user_type_id
WHERE d.distribution_ordinal = 1 AND c.column_id = d.column_id
;

F. List common information about tables.

The following query joins sys.tables, sys.schemas, sys.pdw_table_distribution_properties, sys.columns, sys.types, and sys.pdw_column_distribution_properties to return common information about tables and their columns.

Copy image Copy Code
SELECT t.name, t.object_id, s.name,  tdp.distribution_policy_desc,
  c.name, c.column_id, ty.name, ty.max_length, ty.precision, ty.scale,
  cdp.distribution_ordinal
FROM sys.tables AS t
JOIN sys.schemas AS s
     ON t.schema_id = s.schema_id
JOIN sys.pdw_table_distribution_properties AS tdp
    ON t.object_id = tdp.object_id
JOIN sys.columns AS c
    ON t.object_id = c.object_id
JOIN sys.types AS ty
    ON c.user_type_id = ty.user_type_id
JOIN sys.pdw_column_distribution_properties AS cdp
    ON c.object_id = cdp.object_id AND c.column_id = cdp.column_id
ORDER BY t.name, c.column_id;

[TOP]

Partitioned Tables

A. Get boundary information for each partition

The following example returns the partition numbers, boundary values, boundary value types, and rows per boundary for a partitioned table. This example uses the FactResellerSales table in the AdventureWorksPDW2012 database.

Copy image Copy Code
USE
    AdventureWorksPDW2012
  ;

-- Obtain partition numbers, boundary values, boundary value types,
-- and rows per boundary for the partitions in the FactResellerSales
-- table of the
    AdventureWorksPDW2012
   database.
SELECT sp.partition_number, prv.value AS boundary_value,
lower(sty.name) AS boundary_value_type, sp.rows
FROM sys.tables AS st
JOIN sys.indexes AS si
    ON st.object_id = si.object_id AND si.index_id <2
JOIN sys.partitions AS sp
    ON sp.object_id = st.object_id AND sp.index_id = si.index_id
JOIN sys.partition_schemes AS ps
    ON ps.data_space_id = si.data_space_id
JOIN sys.partition_range_values AS prv
    ON prv.function_id = ps.function_id
JOIN sys.partition_parameters AS pp
    ON pp.function_id = ps.function_id
JOIN sys.types AS sty
    ON sty.user_type_id = pp.user_type_id
        AND prv.boundary_id = sp.partition_number
WHERE st.object_id =
    (SELECT object_id
     FROM sys.objects
     WHERE name = 'FactResellerSales')
ORDER BY sp.partition_number
;

[TOP]

Database and Table Size

Determining the size of table

DBCC PDW_SHOWSPACEUSED is useful for gathering space information.

Copy image Copy Code
DBCC PDW_SHOWSPACEUSED ( "AdventureWorksPDW2012.dbo.DimCustomer" );

B. Finding the number of rows in each partition

The following query returns the number of rows on each partition.

Copy image Copy Code
SELECT t.name, pnp.index_id, pnp.partition_id, pnp.rows,
    pnp.data_compression_desc, pnp.pdw_node_id
FROM sys.pdw_nodes_partitions AS pnp
JOIN sys.pdw_nodes_tables AS NTables
    ON pnp.object_id = NTables.object_id
    AND pnp.pdw_node_id = NTables.pdw_node_id
JOIN sys.pdw_table_mappings AS TMap
    ON NTables.name = TMap.physical_name
JOIN sys.tables AS t
    ON TMap.object_id = t.object_id
WHERE t.name = 'DimCustomer'
ORDER BY t.name, pnp.index_id, pnp.partition_id;

C. View the size of columnstore indexes

The following example joins the sys.pdw_nodes_column_store_row_groups table to other system tables to return information about specific tables. The calculated PercentDeletedRows column is an estimate of the efficiency of the row group; how many deleted rows are consuming space. To find information on a single table remove the comment hyphens in front of the WHERE clause and provide a table name.

Copy image Copy Code
SELECT IndexMap.object_id,
  object_name(IndexMap.object_id) AS LogicalTableName,
  i.name AS LogicalIndexName, IndexMap.index_id, NI.type_desc,
  IndexMap.physical_name AS PhyIndexNameFromIMap,
  CSRowGroups.*,
  100*(ISNULL(deleted_rows,0))/total_rows AS PercentDeletedRows
FROM sys.tables AS t
JOIN sys.indexes AS i
    ON t.object_id = i.object_id
JOIN sys.pdw_index_mappings AS IndexMap
    ON i.object_id = IndexMap.object_id
    AND i.index_id = IndexMap.index_id
JOIN sys.pdw_nodes_indexes AS NI
    ON IndexMap.physical_name = NI.name
    AND IndexMap.index_id = NI.index_id
JOIN sys.pdw_nodes_column_store_row_groups AS CSRowGroups
    ON CSRowGroups.object_id = NI.object_id
    AND CSRowGroups.pdw_node_id = NI.pdw_node_id
    AND CSRowGroups.index_id = NI.index_id
--WHERE t.name = '<table_name>'
ORDER BY object_name(i.object_id), i.name, IndexMap.physical_name, pdw_node_id;

Collations

A. View columns with Latin1_General_100_CI_AS_KS_WS collation

Copy image Copy Code
/* View all columns in DimProduct having the
   Latin1_General_100_CI_AS_KS_WS collation */
USE
    AdventureWorksPDW2012
  ;
SELECT a.name AS 'Table Name', b.name AS 'Column Name',
b.collation_name AS 'Collation Name'
FROM sys.tables AS a
JOIN sys.columns AS b
    ON a.object_id=b.object_id
WHERE b.collation_name = 'Latin1_General_100_CI_AS_KS_WS'
AND a.name = 'DimProduct';

[TOP]

Appliance Health and State

A. Query for a DMV on all Compute nodes.

This query returns the sys.dm_pdw_os_threads DMV on all Compute nodes. The results contain all rows of the DMV from each Compute node.

Copy image Copy Code
SELECT * FROM sys.dm_pdw_os_threads;

B. Query for a DMV on a subset of Compute nodes

This query returns the sys.dm_pdw_os_threads DMV on Compute nodes 2, 3, and 4. The results contain all rows of the DMV from Compute nodes 2, 3, and 4.

Copy image Copy Code
SELECT * FROM sys.dm_pdw_os_threads
WHERE pdw_node_id IN (2, 3, 4);

[TOP]

Security Information

These queries show information about who can access databases and database objects. The graphic shows the relationships of the system views used to gather security information.

APS security related system tables

A. List who can access a database

While in a database, execute the following statement to list the users and roles with access to that database.

Copy image Copy Code
SELECT name, type_desc
FROM sys.database_principals
ORDER BY type_desc;

B. Determine the members of database roles

Execute the following statement to show the members of the database roles.

Copy image Copy Code
SELECT 'Database Role' AS Type, Role.name AS RoleName,
Member.name AS MemberName
FROM sys.database_role_members AS DRM
JOIN sys.database_principals AS Member
    ON DRM.member_principal_id = Member.principal_ID
RIGHT OUTER JOIN sys.database_principals AS Role
    ON DRM.role_principal_id = Role.principal_id
WHERE Role.type_desc = 'DATABASE_ROLE'
ORDER BY RoleName, MemberName;

C. Determine the members of server roles

Execute the following statement to show the members of the server roles.

Copy image Copy Code
SELECT 'Server Role' AS Type, Role.name AS RoleName,
Member.name AS MemberName
FROM sys.server_role_members AS SRM
JOIN sys.server_principals AS Member
    ON SRM.member_principal_id = Member.principal_ID
RIGHT OUTER JOIN sys.server_principals AS Role
    ON SRM.role_principal_id = Role.principal_id
WHERE Role.type_desc = 'SERVER_ROLE'
ORDER BY Type, Role.name;

D. List the permissions explicitly granted or denied in the database

Execute the following query to return the permissions that were explicitly granted or denied to a user or role. This list does not include access that is derived from the fixed server roles (e.g. sysadmin) or the fixed database roles (e.g. db_datareader). When not 0, the major_id value can be used to lookup the names of objects in other system tables.

Copy image Copy Code
SELECT Prin.name, state_desc, permission_name, class_desc, major_id
FROM sys.database_principals AS Prin
JOIN sys.database_permissions AS Perm
    ON Prin.principal_id = Perm.grantee_principal_id
ORDER BY name;

Workload Management

For more information, about workload management, see Workload Management (SQL Server PDW)

A. Display the members of each resource class

CPU, memory, and concurrency resources are allocate to requests according to login membership in resource class server roles. The following query shows the membership in resource classes for each login.

Copy image Copy Code
SELECT l.name AS [member], r.name AS [server role]
FROM sys.server_role_members AS rm
JOIN sys.server_principals AS l
    ON l.principal_id = rm.member_principal_id
JOIN sys.server_principals AS r
    ON r.principal_id = rm.role_principal_id
WHERE l.[type] = 'S'
    AND r.[type] = 'R'
    AND r.name IN ('mediumrc', 'largerc', 'xlargerc' );

B. Find the number of active requests for each resource group

Run the following query to view the number of active requests for each resource group. The sys.dm_pdw_nodes_resource_governor_workload_groups provides the number of active requests per resource group. The sys.dm_pdw_nodes_exec_sessions DMV provides the number of sessions for each group. The LEFT OUTER JOIN narrows the results to only show the session count for resource groups instead of all groups

Copy image Copy Code
SELECT g.group_id, g.name AS GroupName,
COALESCE ( s.SessionCount, 0 ) AS ConnectedSessions,
g.active_request_count AS ActiveRequests
FROM sys.dm_pdw_nodes_resource_governor_workload_groups AS g
LEFT OUTER JOIN
(
    SELECT group_id, SessionCount = COUNT(*)
    FROM sys.dm_pdw_nodes_exec_sessions
    GROUP BY group_id
) AS s
ON g.group_id = s.group_id;

C. Determine if my requests need more memory

This example shows you how to figure out whether your requests might benefit from running with more memory. If a request needs more memory in future runs, you can change the maximum memory by running it under a login that is a member of a larger resource class.

The following query retrieves memory grant information for all of my requests in the current session. The results are listed for each Compute node.

Copy image Copy Code
SELECT pr.request_id, [mem].granted_memory_kb,
[mem].requested_memory_kb, [mem].ideal_memory_kb,
( [mem].ideal_memory_kb - [mem].granted_memory_kb )/( 1024 * 1024 )
    AS requested_memory_gap
-- Get all pdw requests
FROM sys.dm_pdw_exec_requests AS pr
-- For each request_id, add the associated SQL requests on the Compute nodes.
JOIN sys.dm_pdw_sql_requests AS psqlr
    ON psqlr.request_id = pr.request_id
-- Reduce the results to only return requests that are
-- currently running in my session.
JOIN sys.dm_pdw_nodes_exec_requests AS sqlr
    ON sqlr.[session_id] = psqlr.[spid]
        AND sqlr.[pdw_node_id] = psqlr.[pdw_node_id]
-- Add in the memory grant information
JOIN sys.dm_pdw_nodes_exec_query_memory_grants AS [mem]
    ON [mem].[session_id] = psqlr.[spid]
        AND [mem].[pdw_node_id] = psqlr.[pdw_node_id];

D. Determine if there are a high number of query optimizations or suboptimal plan generations.

Copy image Copy Code
SELECT p.pool_id, p.name, g.group_id, g.name,
g.total_query_optimization_count, g.total_suboptimal_plan_generation_count
FROM sys.dm_pdw_nodes_resource_governor_workload_groups AS g
INNER JOIN sys.dm_pdw_nodes_resource_governor_resource_pools AS p
    ON g.pool_id = p.pool_id
ORDER BY g.total_suboptimal_plan_generation_count DESC;

E. Determine Average CPU time per request in each resource pool to date

Copy image Copy Code
SELECT p.pool_id, p.name,
COALESCE ( SUM (t.total_request_count), 0 ) AS total_request_count,
COALESCE (SUM ( t.total_cpu_usage_ms), 0 ) AS total_cpu_usage_ms,
CASE WHEN SUM (t.total_request_count ) > 0
     THEN SUM ( t.total_cpu_usage_ms) / SUM (t.total_request_count )
     ELSE 0
END
FROM sys.dm_pdw_nodes_resource_governor_resource_pools AS p
LEFT OUTER JOIN
(
    SELECT g.pool_id, g.total_request_count, g.total_cpu_usage_ms
    FROM sys.dm_pdw_nodes_resource_governor_workload_groups AS g
    WHERE g.pool_id > 1
) AS t
ON p.pool_id = t.pool_id
GROUP BY p.pool_id, p.name;

Object Definitions

Use the following query to return the definition of a stored procedure.

Copy image Copy Code
SELECT pr.name, modu.definition
FROM sys.procedures AS pr
JOIN sys.sql_modules AS modu
    ON pr.object_ID = modu.object_id
WHERE pr.name = '<proc_name>';

See Also

Reference

System Views (SQL Server PDW)


External Operations Catalog Views (SQL Server PDW)

[image]

External Operations Catalog Views (SQL Server PDW)
See Also Send Feedback

This topic lists the Hadoop catalog views available for use with SQL Server PDW.

Hadoop Catalog Views

sys.external_tables (SQL Server PDW)

See Also

Reference

System Views (SQL Server PDW)


sys.external_file_formats (SQL Server PDW)

[image]

sys.external_file_formats (SQL Server PDW)
See Also Send Feedback

Contains a row for each external file format in SQL Server PDW.

Column Name | Data Type | Description | Range
file_format_id | int | Object ID for the external file format. |
name | sysname | Name of the file format. This is unique for the server. |
format_type | tinyint | The file format type. | DELIMITEDTEXT or RCFILE
field_terminator | nvarchar(10) | For format_type = DELIMITEDTEXT, this is the field terminator. |
string_delimiter | nvarchar(10) | For format_type = DELIMITEDTEXT, this is the string delimiter. |
date_format | nvarchar(50) | For format_type = DELIMITEDTEXT, this is the user-defined date and time format. |
use_type_default | bit | Specifies how to handle missing values when PolyBase is importing data from HDFS text files into SQL Server PDW. | 0 – store missing values as the string 'NULL'. 1 – store missing values as the column default value.
serde_method | nvarchar(255) | For format_type – RCFILE, this is the serialization/deserialization method. |
row_terminator | nvarchar(10) | For format_type = DELIMITEDTEXT, this is the character string that terminates each row in the external Hadoop file. | Always '\n'.
encoding | nvarchar(10) | For format_type = DELIMITEDTEXT, this is the encoding method for the external Hadoop file. | Always 'UTF8'.
data_compression | nvarchar(255) | For format_type = DELIMITEDTEXT, this is the data compression method. | Always NULL. Not supported in this release.

See Also

Reference

CREATE EXTERNAL FILE FORMAT (SQL Server PDW)
System Views (SQL Server PDW)


sys.external_tables (SQL Server PDW)

[image]

sys.external_tables (SQL Server PDW)
See Also Send Feedback

Contains a row for each external table in SQL Server PDW.

Column Name | Data Type | Description | Range
<inherited columns> |  | For a list of columns that this view inherits, see sys.objects. |
max_column_id_used | int | Maximum column ID for the lifetime of the table. | Always 0.
uses_ansi_nulls | bit | Table was created with the SET ANSI_NULLS database option ON. | Always 1 --- true.
data_source_id | int | Object ID for the external data source. |
file_format_id | int | Object ID for the external file format. | NULL if the data source is not a Hadoop data source.
location | nvarchar(4000) | File and folder path of the external data in Hadoop. | NULL if the data source is not a Hadoop data source.
reject_type | tinyint | The way rejected rows are counted when loading external data. | Value – the number of rejected rows. Percentage – the percentage of rejected rows.
reject_value | float | For reject_type = value, this is the number of row rejections to allow before failing the load. For reject_type = percentage, this is the percentage of row rejections to allow before failing the load. | User-defined.
reject_sample_value | int | For reject_type = percentage, this is the number of rows to load, either successfully or unsuccessfully, before calculating the percentage of rejected rows. | NULL if reject_type = value.

See Also

Reference

System Views (SQL Server PDW)


sys.external_data_sources (SQL Server PDW)

[image]

sys.external_data_sources (SQL Server PDW)
See Also Send Feedback

Contains a row for each external data source in SQL Server PDW.

Column Name | Data Type | Description | Range
data_source_id | int | Object ID for the external data source. |
name | sysname | Name of the external data source. This is unique for the server. |
location | nvarchar(4000) | The connection string, which includes the protocol, IP address, and port for the external data source.. |
type_desc | nvarchar(255) | Data source type displayed as a string. | Hadoop or RDBMS
type | tinyint | Data source type displayed as a number. | 0 - Hadoop
job_tracker_location | nvarchar(4000) | IP and port location of the Hadoop job tracker. This is used for submitting a job on the external data source. |

See Also

Reference

CREATE EXTERNAL FILE FORMAT (SQL Server PDW)
System Views (SQL Server PDW)


sys.dm_pdw_hadoop_operations (SQL Server PDW)

[image]

sys.dm_pdw_hadoop_operations (SQL Server PDW)
See Also Send Feedback

Contains a row for each map-reduce job that is pushed down to Hadoop as part of running a SQL Server PDW query on an external Hadoop table. Each map-reduce job represents one of the predicates in the query. This is only used when predicate pushdown is enabled for queries on Hadoop external tables.

For more information about predicate pushdown, see Queries on External Tables With Predicate Pushdown (SQL Server PDW).

Column Name | Data Type | Description | Range
request_id | nvarchar(32) | ID for this external Hadoop operation. | Same as ID in sys.dm_pdw_exec_requests (SQL Server PDW).
step_index | int | Index of the query step that refers to this Hadoop operation. | Same as step_index in sys.dm_pdw_request_steps (SQL Server PDW).
operation_type | nvarchar(255) | Describes the type of external operation. | 'External Hadoop Operation'
operation_name | nvarchar(4000) | The job ID for a map-reduce job. This is returned by Hadoop after SQL Server PDW submits the job. |
map_progress | float | The percentage of input data that has been consumed so far by the map job. | A floating point number between, and including, 0 and 100.
reduce_progress | int | The percentage of the reduce job that has completed.. | A floating point number between, and including, 0 and 100.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_dms_external_work (SQL Server PDW)

[image]

sys.dm_pdw_dms_external_work (SQL Server PDW)
See Also Send Feedback

SQL Server PDW system view that holds information about all Data Movement Service (DMS) steps for external operations.

Column Name | Data Type | Description | Range
request_id | nvarchar(32) | Query that is using this DMS worker. request_id, step_index, and dms_step_index form the key for this view. | Same as request_id in sys.dm_pdw_exec_requests (SQL Server PDW)
step_index | int | Query step that is invoking this DMS worker. request_id, step_index, and dms_step_index form the key for this view. | Same as step_index in sys.dm_pdw_request_steps (SQL Server PDW)
dms_step_index | int | Current step in the DMS plan. request_id, step_index, and dms_step_index form the key for this view. | Same as dms___step_index insys.dm_pdw_dms_workers (SQL Server PDW)
pdw_node_id | int | Node that is running the DMS worker. | Same as node_id in sys.dm_pdw_nodes (SQL Server PDW).
type | nvarchar(60) | Type of external operation this node is running. FILE SPLIT is an operation on an external Hadoop file that has been split into multiple smaller falls. | 'FILE SPLIT'
work_id | int | The file split ID. | Greater than or equal to 0. Unique per Compute node.
input_name | nvarchar(60) | String name for the input being read. | For a Hadoop file, this is the Hadoop file name.
read_location | bigint | Offset of read location. |
estimated_bytes_processed | bigint | Number of bytes processed by this worker. | Greater than or equal to 0.
length | bigint | Number of bytes in the file split. For Hadoop, this is the size of the HDFS block. | User-defined. The default is 64 MB.
status | nvarchar(32) | State of the worker. | Pending, Processing, Done, Failed, Aborted
start_time | datetime | Time at which execution of this worker started. | Greater than or equal to start time of the query step this worker belongs to. See sys.dm_pdw_request_steps (SQL Server PDW).
end_time | datetime | Time at which execution ended, failed, or was cancelled. | NULL for ongoing or queued workers. Otherwise, greater than start_time.
total_elapsed_time | int | Total time spent in execution, in milliseconds. | Greater than or equal to 0. If total_elapsed_time exceeds the maximum value for an integer, total_elapsed_time will continue to be the maximum value. This condition will generate the warning “The maximum value has been exceeded.” The maximum value in milliseconds is equivalent to 24.8 days.

For information about the maximum rows retained by this view, see Maximum System View Values.

See Also

Reference

System Views (SQL Server PDW)


SQL Server PDW Catalog Views (SQL Server PDW)

[image]

SQL Server PDW Catalog Views (SQL Server PDW)
See Also Send Feedback

This topic lists the SQL Server PDW catalog views. These views provide information about the MPP system.

SQL Server PDW Catalog Views

All SQL Server PDW catalog views begin with sys.pdw.

sys.pdw_diag_events (SQL Server PDW)

sys.pdw_diag_event_properties (SQL Server PDW)

sys.pdw_diag_sessions (SQL Server PDW)

sys.pdw_distributions (SQL Server PDW)

sys.pdw_health_alerts (SQL Server PDW)

sys.pdw_health_components (SQL Server PDW)

sys.pdw_health_component_groups (SQL Server PDW)

sys.pdw_health_component_properties (SQL Server PDW)

sys.pdw_health_component_status_mappings (SQL Server PDW)

sys.pdw_index_mappings (SQL Server PDW)

sys.pdw_loader_backup_runs (SQL Server PDW)

sys.pdw_loader_backup_run_details (SQL Server PDW)

sys.pdw_loader_run_stages (SQL Server PDW)

sys.pdw_nodes_column_store_dictionaries (SQL Server PDW)

sys.pdw_nodes_column_store_row_groups (SQL Server PDW)

sys.pdw_nodes_column_store_segments (SQL Server PDW)

sys.pdw_nodes_columns (SQL Server PDW)

sys.pdw_nodes_indexes (SQL Server PDW)

sys.pdw_nodes_partitions (SQL Server PDW)

sys.pdw_nodes_pdw_physical_databases (SQL Server PDW)

sys.pdw_nodes_tables (SQL Server PDW)

sys.pdw_table_distribution_properties (SQL Server PDW)

sys.pdw_table_mappings (SQL Server PDW)

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_column_distribution_properties (SQL Server PDW)

[image]

sys.pdw_column_distribution_properties (SQL Server PDW)
See Also Send Feedback

Holds distribution information for columns.

Column Name | Data Type | Description | Range
object_id | int | ID of the object to which the column belongs. |
column_id | int | ID of the column. |
distribution_ordinal | tinyint | Ordinal (1-based) within set of distribution. | 0 = Not a distribution column. 1 = SQL Server PDW is using this column to distribute the parent table.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_database_mappings (SQL Server PDW)

[image]

sys.pdw_database_mappings (SQL Server PDW)
See Also Send Feedback

Maps the database_ids of databases to the physical name used on Compute nodes, and provides the principal id of the database owner on the system. Join sys.pdw_database_mappings to sys.databases and sys.pdw_nodes_pdw_physical_databases.

Column Name | Data Type | Description | Range
physical_name | nvarchar(36) | The physical name for the database on the Compute nodes. physical_name and database_id form the key for this view. |
database_id | int | The object ID for the database. See sys.databases (SQL Server PDW). physical_name and database_id form the key for this view. |

Example

The following example joins sys.pdw_database_mappings to other system tables to show how databases are mapped.

Copy image Copy Code
SELECT DB.database_id, DB.name, Map.*, Phys.*
FROM sys.databases AS DB
JOIN sys.pdw_database_mappings AS Map
    ON DB.database_id = Map.database_id
JOIN sys.pdw_nodes_pdw_physical_databases AS Phys
    ON Map.physical_name = Phys.physical_name
ORDER BY DB.database_id, Phys.pdw_node_id;

See Also

Reference

System Views (SQL Server PDW)
sys.pdw_index_mappings (SQL Server PDW)
sys.pdw_table_mappings (SQL Server PDW)
sys.pdw_nodes_pdw_physical_databases (SQL Server PDW)


sys.pdw_diag_event_properties (SQL Server PDW)

[image]

sys.pdw_diag_event_properties (SQL Server PDW)
See Also Send Feedback

Holds information about which properties are associated with diagnostic events.

Column Name | Data Type | Description | Range
event_name | nvarchar(255) | Name of the specific diagnostics event. |
property_name | nvarchar(255) | Name of a property of the event. |

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_diag_sessions (SQL Server PDW)

[image]

sys.pdw_diag_sessions (SQL Server PDW)
See Also Send Feedback

Holds information regarding the various diagnostic sessions that have been created on the system.

Column Name | Data Type | Description | Range
name | nvarchar(255) | Name of the diagnostics session. Key for this view. |
xml_data | nvarchar(4000) | XML payload describing the session. |
is_active | bit | Flag indicating whether the flag is active. |
host_address | nvarchar(255) | Address of the machine hosting the session definition (Control node). |
principal_id | int | ID of the user that created the session at the database level. |
database_id | int | ID of the database that is the scope of the diagnostic session.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_diag_events (SQL Server PDW)

[image]

sys.pdw_diag_events (SQL Server PDW)
See Also Send Feedback

Holds information about events that can be included in diagnostic sessions on the system.

Column Name | Data Type | Description | Range
name | nvarchar(255) | Name of the specific diagnostics event. |
source | nvarchar(255) | Source of the event (engine, general, dms, etc.) |
is_enabled | bit | Whether the event is being published. |

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_distributions (SQL Server PDW)

[image]

sys.pdw_distributions (SQL Server PDW)
See Also Send Feedback

Holds information about the distributions on the appliance. It lists one row per appliance distribution.

Column Name | Data Type | Description | Range
distribution_id | int | Unique numeric id associated with the distribution. Key for this view. | 1 to the number of Compute nodes in appliance multiplied by the number of distributions per Compute node.
pdw_node_id | int | ID of the node this distribution is on. | See pdw_node_id in sys.dm_pdw_nodes (SQL Server PDW).
name | nvarchar(32) | String identifier associated with the distribution, used as a suffix on distributed tables. | String composed of 'A-Z','a-z','0-9','_','-'.
position | int | Position of the distribution within a node respective to other distributions on that node. | 1 to the number of distributions per node.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_health_alerts (SQL Server PDW)

[image]

sys.pdw_health_alerts (SQL Server PDW)
See Also Send Feedback

Stores properties for the different alerts that can occur on the system; this is a catalog table for alerts.

Column Name | Data Type | Description | Range
alert_id | int | Unique identifier of the alert. Key for this view. | NOT NULL
component_id | int | ID of the component this alert applies to. The component is a general component identifier, such as "Power Supply," and is not specific to an installation. See sys.pdw_health_components (SQL Server PDW). | NOT NULL
alert_name | nvarchar(255) | Name of the alert. | NOT NULL
state | nvarchar(32) | State of the alert. | NOT NULL Possible values: 'Operational' 'NonOperational' 'Degraded' 'Failed'
severity | nvarchar(32) | Severity of the alert. | NOT NULL Possible values: 'Informational' 'Warning' 'Error'
type | nvarchar(32) | Type of alert. | NOT NULL Possible values: StatusChange - The device status has changed. Threshold - A value has exceeded the threshold value.
description | nvarchar(4000) | Description of the alert. | NOT NULL
condition | nvarchar(255) | Used when type = Threshold. Defines how the alert threshold is calculated. | NULL
status | nvarchar(32) | Alert status | NULL
condition_value | bit | Indicates whether the alert is allowed to occur during system operation. | NULL Possible values 0 - alert is not generated. 1 - alert is generated.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_health_component_groups (SQL Server PDW)

[image]

sys.pdw_health_component_groups (SQL Server PDW)
See Also Send Feedback

Stores information about logical groupings of components and devices.

Column Name | Data Type | Description | Range
group_id | int | Unique identifier for components and devices. Key for this view. | NOT NULL
group_name | nvarchar(255) | Logical group name for the components and devices. | NOT NULL

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_health_component_properties (SQL Server PDW)

[image]

sys.pdw_health_component_properties (SQL Server PDW)
See Also Send Feedback

Stores properties that describe a device. Some properties show device status and some properties describe the device itself.

Column Name | Data Type | Description | Range
property_id | int | Unique identifier of the property of a component. property_id and component_id form the key for this view. | NOT NULL
component_id | int | The ID of the component. See sys.pdw_health_components (SQL Server PDW). property_id and component_id form the key for this view. | NOT NULL
property_name | nvarchar(255) | Name of the property. | NOT NULL
physical_name | nvarchar(32) | Property name as defined by the manufacturer. | NOT NULL
is_key | bit | Determines whether the device instance is unique or not unique. | NOT NULL 0 - Device instance is unique. 1 - Device instance is not unique.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_health_component_status_mappings (SQL Server PDW)

[image]

sys.pdw_health_component_status_mappings (SQL Server PDW)
See Also Send Feedback

Defines the mapping between the SQL Server PDW component statuses and the manufacturer-defined component names.

Column Name | Data Type | Description | Range
property_id | int | Unique identifier of the property. property_id, component_id, and physical_name form the key for this view. | NOT NULL
component_id | int | The ID of the component. See sys.pdw_health_components (SQL Server PDW). property_id, component_id, and physical_name form the key for this view. | NOT NULL
physical_name | nvarchar(32) | Property name as defined by the manufacturer. property_id, component_id, and physical_name form the key for this view. | NOT NULL
logical_name | nvarchar(255) | Property name as defined by SQL Server PDW. | NOT NULL 0 - Device instance is unique. 1 - Device instance is not unique.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_health_components (SQL Server PDW)

[image]

sys.pdw_health_components (SQL Server PDW)
See Also Send Feedback

Stores information about all components and devices that exist in the system. These include hardware, storage devices, and network devices.

Column Name | Data Type | Description | Range
component_id | int | Unique identifier of a component or device. Key for this view. | NOT NULL
group_id | Int | The logical component group to which this component belongs. See sys.pdw_health_component_groups (SQL Server PDW). | NOT NULL
component_name | nvarchar(255) | Name of the component. | NOT NULL

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_index_mappings (SQL Server PDW)

[image]

sys.pdw_index_mappings (SQL Server PDW)
See Also Send Feedback

Maps the logical indexes to the physical name used on Compute nodes as reflected by a unique combination of object_id of the table holding the index and the index_id of a particular index within that table.

Column Name | Data Type | Description | Range
object_id | int | The object ID for the logical table on which this index exists. See sys.objects (SQL Server PDW). physical_name and object_id form the key for this view. |
index_id | nvarchar(32) | The ID for the index. See sys.indexes (SQL Server PDW). |
physical_name | nvarchar(36) | The name of the index in the databases on the Compute nodes. physical_name and object_id form the key for this view. |

See Also

Reference

System Views (SQL Server PDW)
sys.pdw_table_mappings (SQL Server PDW)
sys.pdw_database_mappings (SQL Server PDW)


sys.pdw_loader_backup_run_details (SQL Server PDW)

[image]

sys.pdw_loader_backup_run_details (SQL Server PDW)
See Also Send Feedback

Contains further detailed information, beyond the information in sys.pdw_loader_backup_runs (SQL Server PDW), about ongoing and completed backup, restore, and load operations in SQL Server PDW. The information persists across system restarts.

Column Name | Data Type | Description | Range
run_id | int | Unique identifier for a specific backup or restore run. run_id and pdw_node_id form the key for this view. |
pdw_node_id | int | Unique identifier of an appliance node for which this record holds details. run_id and pdw_node_id form the key for this view. | See node_id in sys.dm_pdw_nodes (SQL Server PDW).
status | nvarchar(16) | The current status of the run. | 'CANCELLED', 'COMPLETED', 'FAILED', 'QUEUED', 'RUNNING'
start_time | datetime | Time at which the operation started on this particular node. |
end_time | datetime | Time at which the operation ended on this particular node, if any. |
total_elapsed_time | int | Total time the operation has been running on this particular node. | If total_elapsed_time exceeds the maximum value for an integer (24.8 days in milliseconds), it will cause materialization failure due to overflow. The maximum value in milliseconds is equivalent to 24.8 days.
progress | int | Progress of the operation expressed as a percentage. | 0 to 100

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_loader_backup_runs (SQL Server PDW)

[image]

sys.pdw_loader_backup_runs (SQL Server PDW)
See Also Send Feedback

Contains information about ongoing and completed backup, restore, and load operations in SQL Server PDW. The information persists across system restarts.

Column Name

Data Type

Description

Range

run_id

int

Unique identifier for a specific backup, restore, or load run.

Key for this view.

name

nvarchar(255)

Null for load. Optional name for backup or restore.

submit_time

datetime

Time the request was submitted.

start_time

datetime

Time the operation started.

end_time

datetime

Time the operation completed, failed, or was cancelled.

total_elapsed_time

int

Total time elapsed between start_time and current time, or between start_time and end_time for completed, cancelled, or failed runs.

If total_elapsed_time exceeds the maximum value for an integer (24.8 days in milliseconds), it will cause materialization failure due to overflow.

The maximum value in milliseconds is equivalent to 24.8 days.

operation_type

nvarchar(16)

The load type.

'BACKUP', 'LOAD', 'RESTORE'

mode

nvarchar(16)

The mode within the run type.

For operation_type = BACKUP

  • DIFFERENTIAL

  • FULL

For operation_type = LOAD

  • APPEND

  • RELOAD

  • UPSERT

For operation_type = RESTORE

  • DATABASE

  • HEADER_ONLY

database_name

nvarchar(255)

Name of the database that is the context of this operation

table_name

nvarchar(255)

Information not available.

Principal_id

int

ID of the user requesting the operation.

session_id

nvarchar(32)

ID of the session performing the operation.

See session_id in sys.dm_pdw_exec_sessions (SQL Server PDW).

request_id

nvarchar(32)

ID of the request performing the operation. For loads, this is the current or last request associated with this load..

See request_id in sys.dm_pdw_exec_requests (SQL Server PDW).

status

nvarchar(16)

Status of the run.

'CANCELLED','COMPLETED','FAILED','QUEUED','RUNNING'

progress

int

Percentage completed.

0 to 100

command

nvarchar(4000)

Full text of the command submitted by the user.

Will be truncated if longer than 4000 characters (counting spaces).

rows_processed

bigint

Number of rows processed as part of this operation.

rows_rejected

bigint

Number of rows rejected as part of this operation.

rows_inserted

bigint

Number of rows inserted into the database table(s) as part of this operation.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_loader_run_stages (SQL Server PDW)

[image]

sys.pdw_loader_run_stages (SQL Server PDW)
See Also Send Feedback

Contains information about ongoing and completed load operations in SQL Server PDW. The information persists across system restarts.

Column Name | Data Type | Description | Range
run_id | int | Unique identifier of a loader run. |
stage | nvarchar(30) | The current stage for the run. | 'CREATE_STAGING', 'DMS_LOAD', 'LOAD_INSERT', 'LOAD_CLEANUP'
request_id | nvarchar(32) | ID of the request running this stage. |
status | nvarchar(16) | Status of this phase. |
start_time | datetime | Time at which the stage was started. |
end_time | datetime | Time at which the stage ended, if any. | NULL if not started or in progress.
total_elapsed_time | int | Total time this stage spent (or spent so far) running. | If total_elapsed_time exceeds the maximum value for an integer (24.8 days in milliseconds), it will cause materialization failure due to overflow. The maximum value in milliseconds is equivalent to 24.8 days.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_nodes_column_store_dictionaries (SQL Server PDW)

[image]

sys.pdw_nodes_column_store_dictionaries (SQL Server PDW)
See Also Send Feedback

Contains a row for each dictionary used in columnstore indexes. Dictionaries are used to encode some, but not all data types, therefore not all columns in a columnstore index have dictionaries. A dictionary can exist as a primary dictionary (for all segments) and possibly for other secondary dictionaries used for a subset of the column's segments.

Column name | Data type | Description
partition_id | bigint | Indicates the partition ID. Is unique within a database.
hobt_id | bigint | ID of the heap or B-tree index (hobt) for the table that has this columnstore index.
column_id | int | ID of the columnstore column.
dictionary_id | int | Id of the dictionary.
version | int | Version of the dictionary format.
type | int | Dictionary type: 1 – Hash dictionary containing int values 2 – Not used 3 – Hash dictionary containing string values 4 – Hash dictionary containing float values
last_id | int | The last data id in the dictionary.
entry_count | bigint | Number of entries in the dictionary.
on_disc_size | bigint | Size of dictionary in bytes.
pdw_node_id | int | Unique identifier of a SQL Server PDW node.

Permissions

Requires VIEW SERVER STATE permission.

See Also

Reference

System Views (SQL Server PDW)
CREATE COLUMNSTORE INDEX (SQL Server PDW)
sys.pdw_nodes_column_store_segments (SQL Server PDW)
sys.pdw_nodes_column_store_row_groups (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Clustered Columnstore Indexes (SQL Server PDW)


sys.pdw_nodes_column_store_row_groups (SQL Server PDW)

[image]

sys.pdw_nodes_column_store_row_groups (SQL Server PDW)
See Also Send Feedback

Provides clustered columnstore index information on a per-segment basis to help the administrator make system management decisions in SQL Server PDW. sys.pdw_nodes_column_store_row_groups has a column for the total number of rows physically stored (including those marked as deleted) and a column for the number of rows marked as deleted. Use sys.pdw_nodes_column_store_row_groups to determine which row groups have a high percentage of deleted rows and should be rebuilt.

Column name | Data type | Description
object_id | int | ID of the underlying table. This is the physical table on the Compute node, not the object_id for the logical table on the Control node. For example, object_id does not match with the object_id in sys.tables. To join with sys.tables, use sys.pdw_index_mappings.
index_id | int | ID of the clustered columnstore index on object_id table.
partition_number | int | ID of the table partition that holds row group row_group_id. You can use partition_number to join this DMV to sys.partitions.
row_group_id | int | ID of this row group. This is unique within the partition.
dellta_store_hobt_id | bigint | The hobt_id for delta row groups, or NULL if the row group type is not delta. A delta row group is a read/write row group that is accepting new records. A delta row group has the OPEN status. A delta row group is still in rowstore format and has not been compressed to columnstore format.
state | tinyint | ID number associated with the state_description. 1 = OPEN 2 = CLOSED 3 = COMPRESSED
state_desccription | nvarchar(60) | Description of the persistent state of the row group: OPEN – A read/write row group that is accepting new records. An open row group is still in rowstore format and has not been compressed to columnstore format. CLOSED – A row group that has been filled, but not yet compressed by the tuple mover process. COMPRESSED – A row group that has filled and compressed.
total_rows | bigint | Total rows physically stored in the row group. Some may have been deleted but they are still stored. The maximum number of rows in a row group is 1,048,576 (hexadecimal FFFFF).
deleted_rows | bigint | Number of rows physically stored in the row group that are marked for deletion. Always 0 for DELTA row groups.
size_in_bytes | int | Combined size, in bytes, of all the pages in this row group. This size does not include the size required to store metadata or shared dictionaries.
pdw_node_id | int | Unique id of a SQL Server PDW node.

Remarks

Returns one row for each columnstore row group for each table having a clustered or nonclustered columnstore index.

Use sys.pdw_nodes_column_store_row_groups to determine the number of rows included in the row group and the size of the row group.

When the number of deleted rows in a row group grows to a large percentage of the total rows, the table becomes less efficient. Rebuild the columnstore index to reduce the size of the table, reducing the disk I/O required to read the table. To rebuild the columnstore index use the REBUILD option of the ALTER INDEX statement.

The updateable columnstore first inserts new data into an OPEN rowgroup, which is in rowstore format, and is also sometimes referred to as a delta table. Once an open rowgroup is full, its state changes to CLOSED. A closed rowgroup is compressed into columnstore format by the tuple mover and the state changes to COMPRESSED. The tuple mover is a background process that periodically wakes up and checks whether there are any closed rowgroups that are ready to compress into a columnstore rowgroup. The tuple mover also deallocates any rowgroups in which every row has been deleted. Deallocated rowgroups are marked as RETIRED. To run tuple mover immediately, use the REORGANIZE option of the ALTER INDEX statement.

When a columnstore row group has filled, it is compressed, and stops accepting new rows. When rows are deleted from a compressed group, they remain but are marked as deleted. Updates to a compressed group are implemented as a delete from the compressed group, and an insert to an open group.

Permissions

Requires VIEW SERVER STATE permission.

Examples

The following example joins the sys.pdw_nodes_column_store_row_groups table to other system tables to return information about specific tables. The calculated PercentFull column is an estimate of the efficiency of the row group. To find information on a single table remove the comment hyphens in front of the WHERE clause and provide a table name.

Copy image Copy Code
SELECT IndexMap.object_id,
  object_name(IndexMap.object_id) AS LogicalTableName,
  i.name AS LogicalIndexName, IndexMap.index_id, NI.type_desc,
  IndexMap.physical_name AS PhyIndexNameFromIMap,
  CSRowGroups.*,
  100*(ISNULL(deleted_rows,0))/total_rows AS PercentDeletedRows
FROM sys.tables AS t
JOIN sys.indexes AS i
    ON t.object_id = i.object_id
JOIN sys.pdw_index_mappings AS IndexMap
    ON i.object_id = IndexMap.object_id
    AND i.index_id = IndexMap.index_id
JOIN sys.pdw_nodes_indexes AS NI
    ON IndexMap.physical_name = NI.name
    AND IndexMap.index_id = NI.index_id
JOIN sys.pdw_nodes_column_store_row_groups AS CSRowGroups
    ON CSRowGroups.object_id = NI.object_id
    AND CSRowGroups.pdw_node_id = NI.pdw_node_id
AND CSRowGroups.index_id = NI.index_id
--WHERE t.name = '<table_name>'
ORDER BY object_name(i.object_id), i.name, IndexMap.physical_name, pdw_node_id;

See Also

Reference

System Views (SQL Server PDW)
CREATE COLUMNSTORE INDEX (SQL Server PDW)
sys.pdw_nodes_column_store_segments (SQL Server PDW)
sys.pdw_nodes_column_store_dictionaries (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Clustered Columnstore Indexes (SQL Server PDW)


sys.pdw_nodes_column_store_segments (SQL Server PDW)

[image]

sys.pdw_nodes_column_store_segments (SQL Server PDW)
See Also Send Feedback

Contains a row for each column in a columnstore index.

Column name | Data type | Description
partition_id | bigint | Indicates the partition ID. Is unique within a database.
hobt_id | bigint | ID of the heap or B-tree index (hobt) for the table that has this columnstore index.
column_id | int | ID of the columnstore column.
segment_id | int | ID of the column segment.
version | int | Version of the column segment format.
encoding_type | int | Type of encoding used for that segment.
row_count | int | Number of rows in the row group.
has_nulls | int | 1 if the column segment has null values.
base_id | bigint | Base value id if encoding type 1 is being used. If encoding type 1 is not being used, base_id is set to 1.
magnitude | float | Magnitude if encoding type 1 is being used. If encoding type 1 is not being used, magnitude is set to 1.
primary__dictionary_id | int | Id of primary dictionary.
secondary_dictionary_id | int | Id of secondary dictionary. Returns -1 if there is no secondary dictionary.
min_data_id | bigint | Minimum data id in the column segment.
max_data_id | bigint | Maximum data id in the column segment.
null_value | bigint | Value used to represent nulls.
on_disk_size | bigint | Size of segment in bytes.
pdw_node_id | int | Unique identifier of a SQL Server PDW note.

Remarks

The following query returns information about segments of a columnstore index.

Transact-SQL Copy image Copy Code
SELECT i.name, p.object_id, p.index_id, i.type_desc,
    COUNT(*) AS number_of_segments
FROM sys.column_store_segments AS s
INNER JOIN sys.partitions AS p
    ON s.hobt_id = p.hobt_id
INNER JOIN sys.indexes AS i
    ON p.object_id = i.object_id
WHERE i.type = 6
GROUP BY i.name, p.object_id, p.index_id, i.type_desc ;

Join sys.pdw_nodes_column_store_segments with other system tables to determine the row count and on-disk size of the segments.

Copy image Copy Code
SELECT o.name, css.hobt_id, css. column_id, css.pdw_node_id, css.row_count, css.on_disk_size
FROM sys.pdw_nodes_column_store_segments AS css
JOIN sys.pdw_nodes_partitions AS pnp
    ON css.partition_id = pnp.partition_id
JOIN sys.pdw_nodes_tables AS part
    ON pnp.object_id = part.object_id
    AND pnp.pdw_node_id = part.pdw_node_id
JOIN sys.pdw_table_mappings AS TMap
    ON part.name = TMap.physical_name
JOIN sys.objects AS o
    ON TMap.object_id = o.object_id
ORDER BY css.hobt_id, css.column_id;

Permissions

Requires VIEW SERVER STATE permission.

See Also

Reference

System Views (SQL Server PDW)
CREATE COLUMNSTORE INDEX (SQL Server PDW)
sys.pdw_nodes_column_store_row_groups (SQL Server PDW)
sys.pdw_nodes_column_store_dictionaries (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Clustered Columnstore Indexes (SQL Server PDW)


sys.pdw_nodes_columns (SQL Server PDW)

[image]

sys.pdw_nodes_columns (SQL Server PDW)
See Also Send Feedback

Shows columns for user-defined tables and user-defined views.

Column Name | Data Type | Description | Range
object_id | int | ID of the object to which this column belongs. |
name | sysname | Name of the column. Unique in object. |
column_id | int | ID of the column. Unique in object. |
system_type_id | tinyint | ID of the system type of the column. |
user_type_id | int | ID of the type of the column as defined by the user. |
max_length | smallint | Maximum length (in bytes) of the column. | Includes -1 (not valid) for unsupported column types.
precision | tinyint | Precision of the column if numeric-based; otherwise, 0. |
scale | tinyint | Scale of column if numeric-based; otherwise, 0. |
collation_name | sysname | Name of the collation of the column if character-based; otherwise, NULL. |
is_nullable | bit | 1 = Column is nullable. |
is_ansi_padded | bit | 1 = Column uses ANSI_PADDING ON behavior if character, binary, or variant. | Always 0.
is_rowguidcol | bit | 1 = Column is a declared ROWGUIDCOL. | Always 0.
is_identity | bit | 1 = Column has identity values. | Always 0.
is_computed | bit | 1 = Column is a computed column. | Always 0.
is_filestream | bit | 1 = Column is a FILESTREAM column. | Always 0.
is_replicated | bit | 1 = Column is replicated. | Always 0.
is_non_sql_subscribed | bit | 1 = Column has a non-SQL subscriber. | Always 0.
is_merge_published | bit | 1 = Column is merge-published. | Always 0.
is_dts_replicated | bit | 1 = Column is replicated by using SSIS. | Always 0.
is_xml_document | bit | 1 = Content is a complete XML document. | Always 0.
xml_collection_id | int | 0 = No XML schema collection. | Always 0.
default_object_id | int | ID of the default object; 0 = No default. | Always 0.
rule_object_id | int | ID of the stand-alone rule bound to the column. 0 = No stand-alone rule. | Always 0.
is_sparse | bit | 1 = Column is a sparse column. | Always 0.
is_column_set | bit | 1 = Column is a column set. | Always 0.
pdw_node_id | int | Unique identifier of a SQL Server PDW node. | NOT NULL

Permissions

Requires CONTROL SERVER permission.

See Also

Reference

sys.all_columns (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.pdw_nodes_indexes (SQL Server PDW)

[image]

sys.pdw_nodes_indexes (SQL Server PDW)
See Also Send Feedback

Returns indexes for SQL Server PDW.

Column Name | Data Type | Description | Range
object_id | int | id of the object to which this index belongs. |
name | sysname | Name of the index. Name is unique only within the object. NULL = Heap |
index_id | int | id of the index. index_id is unique only within the object. 0 = Heap 1 = Clustered index > 1 = Non-clustered index |
type | tinyint | Type of index: 0 = Heap 1 = Clustered 2 = Non-clustered 5 = Clustered xVelocity memory optimized columnstore index
type_desc | nvarchar(60) | Description of index type: HEAP CLUSTERED NONCLUSTERED CLUSTERED COLUMNSTORE |
is_unique | bit | 0 = Index is not unique. | Always 0.
data_space_id | int | id of the data space for this index. Data space is either a filegroup or partition scheme. 0 = object_id is a table-valued function. |
ignore_dup_key | bit | 0 = IGNORE_DUP_KEY is OFF. | Always 0.
is_primary_key | bit | 1 = Index is part of a PRIMARY KEY constraint. | Always 0.
is_unique_constraint | bit | 1 = Index is part of a UNIQUE constraint. | Always 0.
fill_factor | tinyint | > 0 = FILLFACTOR percentage used when the index was created or rebuilt. 0 = Default value | Always 0.
is_padded | bit | 0 = PADINDEX is OFF. | Always 0.
is_disabled | bit | 1 = Index is disabled. 0 = Index is not disabled. |
is_hypothetical | bit | 0 = Index is not hypothetical. | Always 0.
allow_row_locks | bit | 1 = Index allows row locks. | Always 1.
allow_page_locks | bit | 1 = Index allows page locks. | Always 1.
has_filter | bit | 0 = Index does not have a filter. | Always 0.
filter_definition | nvarchar(max) | Expression for the subset of rows included in the filtered index. | Always NULL.
pdw_node_id | int | Unique identifier of a SQL Server PDW node. | NOT NULL

Permissions

Requires CONTROL SERVER permission.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_nodes_partitions (SQL Server PDW)

[image]

sys.pdw_nodes_partitions (SQL Server PDW)
See Also Send Feedback

Contains a row for each partition of all the tables, and most types of indexes in a SQL Server PDW database. All tables and indexes contain at least one partition, whether or not they are explicitly partitioned.

Column name

Data type

Description

partition_id

bigint

id of the partition. Is unique within a database.

object_id

int

id of the object to which this partition belongs. Every table or view is composed of at least one partition.

index_id

int

id of the index within the object to which this partition belongs.

partition_number

int

1-based partition number within the owning index or heap. For SQL Server PDW, the value of this column is 1.

hobt_id

bigint

ID of the data heap or B-tree that contains the rows for this partition.

rows

bigint

Approximate average number of rows in each table partition. To calculate this value, SQL Server PDW divides the number of rows in the table by the number of partitions in the table.

SQL Server PDW uses statistics, which might be out-of-date, to determine the total number of rows. The statistics are from the most recent run of UPDATE STATISTICS on the table. If UPDATE STATISTICS has not been run on the table, the statistics won’t exist, and SQL Server PDW will use 1000 as the default total number of rows.

To display the number of rows in each partition within each distribution, use DBCC PDW_SHOWPARTITIONSTATS (SQL Server PDW).

data_compression

int

Indicates the state of compression for each partition:

0 = NONE

1 = ROW

2 = PAGE

3 = COLUMNSTORE

data_compression_desc

nvarchar(60)

Indicates the state of compression for each partition. Possible values are NONE, ROW, and PAGE.

pdw_node_id

int

Unique identifier of a SQL Server PDW node.

Permissions

Requires CONTROL SERVER permission.

Examples

The following query returns the number of rows on each partition.

Copy image Copy Code
SELECT o.name, pnp.index_id, pnp.partition_id, pnp.rows,
    pnp.data_compression_desc, pnp.pdw_node_id
FROM sys.pdw_nodes_partitions AS pnp
JOIN sys.pdw_nodes_tables AS NTables
    ON pnp.object_id = NTables.object_id
AND pnp.pdw_node_id = NTables.pdw_node_id
JOIN sys.pdw_table_mappings AS TMap
    ON NTables.name = TMap.physical_name
JOIN sys.objects AS o
    ON TMap.object_id = o.object_id
WHERE o.name = 'myDimCustomer'
ORDER BY o.name, pnp.index_id, pnp.partition_id;

DBCC PDW_SHOWSPACEUSED is also useful for gathering space information.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_nodes_pdw_physical_databases (SQL Server PDW)

[image]

sys.pdw_nodes_pdw_physical_databases (SQL Server PDW)
See Also Send Feedback

Contains a row for each physical database on a compute node. Aggregate physical database information to get detailed information about databases. To combine information, join the sys.pdw_nodes_pdw_physical_databases to the sys.pdw_database_mappings and sys.databases tables.

Column Name | Data Type | Description
database_id | int | The object ID for the database. Note that this value is not same as a database_id in the sys.databases (SQL Server PDW) view.
physical_name | sysname | The physical name for the database on the Shell/Compute nodes. This value is same as a value in the physical_name column in the sys.pdw_database_mappings (SQL Server PDW) view.
pdw_node_id | int | Unique numeric id associated with the node.

Examples

A. Returning

The following query returns the name and ID of each database in master, and the corresponding database name on each compute node.

Copy image Copy Code
SELECT D.database_id AS DBID_in_master, D.name AS UserDatabaseName,
PD.pdw_node_id AS NodeID, DM.physical_name AS PhysDBName
FROM sys.databases AS D
JOIN sys.pdw_database_mappings AS DM
    ON D.database_id = DM.database_id
JOIN sys.pdw_nodes_pdw_physical_databases AS PD
    ON DM.physical_name = PD.physical_name
ORDER BY D.database_id, PD.pdw_node_ID;

B. Using sys.pdw_nodes_pdw_physical_databases to gather detailed object information

The following query shows information about indexes and includes useful information about the database the objects belong to objects in the database.

Copy image Copy Code
SELECT D.name AS UserDatabaseName, D.database_id AS DBIDinMaster,
DM.physical_name AS PhysDBName, PD.pdw_node_id AS NodeID,
IU.object_id, IU.index_id, IU.user_seeks, IU.user_scans, IU.user_lookups, IU.user_updates
FROM sys.databases AS D
JOIN sys.pdw_database_mappings AS DM
    ON D.database_id = DM.database_id
JOIN sys.pdw_nodes_pdw_physical_databases AS PD
    ON DM.physical_name = PD.physical_name
JOIN sys.dm_pdw_nodes_db_index_usage_stats AS IU
    ON PD.database_id = IU.database_id
ORDER BY D.database_id, IU.object_id, IU.index_id, PD.pdw_node_ID;

C. Using sys.pdw_nodes_pdw_physical_databases to determine the encryption state

The following query provides encryption state of the AdventureWorksPDW2012 database.

Copy image Copy Code
WITH dek_encryption_state AS
(
    SELECT ISNULL(db_map.database_id, dek.database_id) AS database_id, encryption_state
    FROM sys.dm_pdw_nodes_database_encryption_keys AS dek
        INNER JOIN sys.pdw_nodes_pdw_physical_databases AS node_db_map
            ON dek.database_id = node_db_map.database_id AND dek.pdw_node_id = node_db_map.pdw_node_id
        LEFT JOIN sys.pdw_database_mappings AS db_map
            ON node_db_map .physical_name = db_map.physical_name
        INNER JOIN sys.dm_pdw_nodes AS nodes
            ON nodes.pdw_node_id = dek.pdw_node_id
    WHERE dek.encryptor_thumbprint <> 0x
)
SELECT TOP 1 encryption_state
       FROM  dek_encryption_state
       WHERE dek_encryption_state.database_id = DB_ID('AdventureWorksPDW2012 ')
       ORDER BY (CASE encryption_state WHEN 3 THEN -1 ELSE encryption_state END) DESC;

See Also

Reference

System Views (SQL Server PDW)
sys.databases (SQL Server PDW)
sys.pdw_database_mappings (SQL Server PDW)


sys.pdw_nodes_tables (SQL Server PDW)

[image]

sys.pdw_nodes_tables (SQL Server PDW)
See Also Send Feedback

Contains a row for each table object that a principal either owns or on which the principal has been granted some permission.

Column Name | Data Type | Description | Range
<inherited columns> |  | For a list of columns that this view inherits, see sys.objects. |
lob_data_space_id | int |  | Always 0.
filestream_data_space_id | int | Data space ID for a FILESTREAM filegroup or Information not available. | NULL
max_column_id_used | int | Maximum column ID used by this table. |
lock_on_bulk_load | bit | Table is locked on bulk load. | TBD
uses_ansi_nulls | bit | Table was created with the SET ANSI_NULLS database option ON. | 1
is_replicated | bit | 1 = Table is published using replication. | 0; replication is not supported.
has_replication_filter | bit | 1 = Table has a replication filter. | 0
is_merge_published | bit | 1 = Table is published using merge replication. | 0; not supported.
is_sync_tran_subscribed | bit | 1 = Table is subscribed using an immediate updating subscription. | 0; not supported.
has_unchecked_assembly_data | bit | 1 = Table contains persisted data that depends on an assembly whose definition changed during the last ALTER ASSEMBLY. Will be reset to 0 after the next successful DBCC CHECKDB or DBCC CHECKTABLE. | 0; no CLR support.
text_in_row_limit | int | 0 = Text in row option is not set. | Always 0.
large_value_types_out_of_row | bit | 1 = Large value types are stored out-of-row. | Always 0.
is_tracked_by_cdc | bit | 1 = Table is enabled for change data capture | Always 0; no CDC support.
lock_escalation | tinyint | The value of the LOCK_ESCALATION option for the table: 2 = AUTO | Always 2.
lock_escalation_desc | nvarchar(60) | A text description of the lock_escalation option. | Always ꞌAUTOꞌ.
pdw_node_id | int | Unique identifier of a SQL Server PDW node. | NOT NULL

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_table_distribution_properties (SQL Server PDW)

[image]

sys.pdw_table_distribution_properties (SQL Server PDW)
See Also Send Feedback

Holds distribution information for tables.

Column Name | Data Type | Description | Range
object_id | int | ID of the table for which thee properties were specified. |
distribution_policy | tinyint | 0 = Undefined 1 = None 2 = Hash 3 = Replicate 4 = Segment Heap | SQL Server PDW returns either 2 or 3.
distribution_policy_desc | nvarchar(60) | UNDEFINED, NONE, HASH, REPLICATE, SEGMENTED_HEAP | SQL Server PDW returns either HASH or REPLICATE.

See Also

Reference

System Views (SQL Server PDW)


sys.pdw_table_mappings (SQL Server PDW)

[image]

sys.pdw_table_mappings (SQL Server PDW)
See Also Send Feedback

Ties user tables to internal object names by object_id.

Column Name | Data Type | Description | Range
physical_name | nvarchar(36) | The physical name for the table. physical_name and object_id form the key for this view. |
object_id | int | The object ID for the table. See sys.objects (SQL Server PDW). physical_name and object_id form the key for this view. |

See Also

Reference

System Views (SQL Server PDW)
sys.pdw_index_mappings (SQL Server PDW)
sys.pdw_database_mappings (SQL Server PDW)


SQL Server PDW Dynamic Management Views (SQL Server PDW)

[image]

SQL Server PDW Dynamic Management Views (SQL Server PDW)
See Also Send Feedback

This topic lists the SQL Server PDW dynamic management views (DMVs). These views provide information about the MPP system.

SQL Server PDW Dynamic Management Views

The list below links to the individual topic for each SQL Server PDW DMV.

All SQL Server PDW DMVs begin with sys.dm_pdw.

sys.dm_pdw_component_health_active_alerts (SQL Server PDW)

sys.dm_pdw_component_health_alerts (SQL Server PDW)

sys.dm_pdw_component_health_status (SQL Server PDW)

sys.dm_pdw_diag_processing_stats (SQL Server PDW)

sys.dm_pdw_dms_cores (SQL Server PDW)

sys.dm_pdw_dms_workers (SQL Server PDW)

sys.dm_pdw_errors (SQL Server PDW)

sys.dm_pdw_exec_requests (SQL Server PDW)

sys.dm_pdw_exec_sessions (SQL Server PDW)

sys.dm_pdw_node_status (SQL Server PDW)

sys.dm_pdw_nodes (SQL Server PDW)

sys.dm_pdw_nodes_database_encryption_keys (SQL Server PDW)

sys.dm_pdw_os_event_logs (SQL Server PDW)

sys.dm_pdw_os_performance_counters (SQL Server PDW)

sys.dm_pdw_os_threads (SQL Server PDW)

sys.dm_pdw_query_stats_xe (SQL Server PDW)

sys.dm_pdw_query_stats_xe_file (SQL Server PDW)

sys.dm_pdw_request_steps (SQL Server PDW)

sys.dm_pdw_resource_waits (SQL Server PDW)

sys.dm_pdw_sql_requests (SQL Server PDW)

sys.dm_pdw_sys_info (SQL Server PDW)

sys.dm_pdw_wait_stats (SQL Server PDW)

sys.dm_pdw_waits (SQL Server PDW)

sys.dm_pdw_lock_waits (SQL Server PDW)

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_component_health_active_alerts (SQL Server PDW)

[image]

sys.dm_pdw_component_health_active_alerts (SQL Server PDW)
See Also Send Feedback

Stores active alerts on SQL Server PDW components.

Column Name | Data Type | Description | Range
pdw_node_id | int | Unique identifier of a SQL Server PDW node. pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
component_id | int | The ID of the component. See sys.pdw_health_components (SQL Server PDW). pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
component_instance_id | nvarchar(255) | pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
alert_id | int | The ID for the alert type. See sys.pdw_health_alerts (SQL Server PDW). pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
alert_instance_id | nvarchar(36) | Identifies an instance of a given alert. pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
current_value | nvarchar(255) | Used when the alert is of type StatusChange. This is the current component status. Value is NULL for alerts of type Threshold. See sys.pdw_health_alerts (SQL Server PDW) for a list of alert types. | NULL
previous_value | nvarchar(255) | Used when the alert is of type StatusChange. This is the previous component status. Value is NULL for alerts of type Threshold. See sys.pdw_health_alerts (SQL Server PDW) for a list of alert types. | NULL
create_time | datetime | Time and date when the alert was generated. | NOT NULL

For information about the maximum rows retained by this view, see the Maximum System View Values section in the Minimum and Maximum Values (SQL Server PDW) topic.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_component_health_alerts (SQL Server PDW)

[image]

sys.dm_pdw_component_health_alerts (SQL Server PDW)
See Also Send Feedback

Stores previously issued alerts on appliance components.

Column Name | Data Type | Description | Range
pdw_node_id | int | Unique identifier of a SQL Server PDW node. pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
component_id | int | The ID of the component. See sys.pdw_health_components (SQL Server PDW). pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
component_instance_id | nvarchar(255) | pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
alert_id | int | The ID for the alert type. See sys.pdw_health_alerts (SQL Server PDW). pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
alert_instance_id | nvarchar(36) | Identifies an instance of a given alert. pdw_node_id, component_id, component_instance_id, alert_id, and alert_instance_id form the key for this view. | NOT NULL
previous_value | nvarchar(255) | Used when the alert is of type StatusChange. This is the previous component status. Value is NULL for alerts of type Threshold. See sys.pdw_health_alerts (SQL Server PDW) for a list of alert types. | NULL
current_value | nvarchar(255) | Used when the alert is of type StatusChange. This is the current component status. Value is NULL for alerts of type Threshold. See sys.pdw_health_alerts (SQL Server PDW) for a list of alert types. | NULL
create_time | datetime | Time and date when the alert was generated. | NOT NULL

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_component_health_status (SQL Server PDW)

[image]

sys.dm_pdw_component_health_status (SQL Server PDW)
See Also Send Feedback

Holds information about the current health of appliance components.

Column Name | Data Type | Description | Range
pdw_node_id | int |  | Not NULL
component_id | int | The ID of the component. See sys.pdw_health_components (SQL Server PDW). pdw_node_id, component_id, property_id, and component_instance_id form the key for this view. | Not NULL
property_id | int | The ID of the property. See sys.pdw_health_component_properties (SQL Server PDW). | NOT NULL
component_instance_id | nvarchar(255) | Identifies an instance of a component. For example, an instance of a CPU might be identified by component_instance_id='CPU1'. pdw_node_id, component_id, property_id, and component_instance_id form the key for this view. | NOT NULL
property_value | nvarchar(255) | The current property value. | NULL
update_time | datetime | The last time the metric was updated. | NOT NULL

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_diag_processing_stats (SQL Server PDW)

[image]

sys.dm_pdw_diag_processing_stats (SQL Server PDW)
See Also Send Feedback

Displays information related to all internal diagnostic events that could be incorporated into diagnostic sessions defined by the administrator. Query this view to understand the statistics behind the diagnostics and eventing subsystems that drive the population of all the other DMVs. There are a group of queues for each process on each node.

Column Name | Data Type | Description
pdw_node_id | int | Appliance node this log is from.
process_id | int | Identifier of the process running submitting this statistic.
target_name | nvarchar(255) | The name of the queue.
queue_size | int | The number of items in the process queue. The queue size is usually 0. A positive number indicates that the system is under stress and is building backlog of events. A positive count in the other columns means system has become corrupted for that particular queue and any related DMVs.
lost_events_count | bigint | The number of events lost.

See Also


sys.dm_pdw_dms_cores (SQL Server PDW)

[image]

sys.dm_pdw_dms_cores (SQL Server PDW)
See Also Send Feedback

Holds information about all DMS services running on the Compute nodes of the appliance. It lists one row per service instance, which is currently one row per node.

Column Name | Data Type | Description | Range
dms_core_id | int | Unique numeric id associated with this DMS core. Key for this view. | Set to the pdw_node_id of the node that this DMS core is running on.
pdw_node_id | int | ID of the node on which this DMS service is running. | See node_id in sys.dm_pdw_nodes (SQL Server PDW).
status | nvarchar(32) | Current status of the DMS service. | Information not available.

For information about the maximum rows retained by this view, see the Maximum System View Values section in the Minimum and Maximum Values (SQL Server PDW) topic.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_dms_workers (SQL Server PDW)

[image]

sys.dm_pdw_dms_workers (SQL Server PDW)
See Also Send Feedback

Holds information about all workers completing DMS steps.

Column Name | Data Type | Description | Range
request_id | nvarchar(32) | Query that this DMS worker is part of. request_id, step_index, and dms_step_index form the key for this view. | See request_id in sys.dm_pdw_exec_requests (SQL Server PDW)
step_index | int | Query step this DMS worker is part of. request_id, step_index, and dms_step_index form the key for this view. | See step_index in sys.dm_pdw_request_steps (SQL Server PDW)
dms_step_index | int | Step in the DMS plan that this worker is running. request_id, step_index, and dms_step_index form the key for this view. |
pdw_node_id | int | Node that the worker is running on. | See node_id in sys.dm_pdw_nodes (SQL Server PDW).
distribution_id | Int | Distribution that the worker is running on, if any. | See distribution_id in sys.pdw_distributions (SQL Server PDW).
type | nvarchar(32) | Type of DMS worker thread this entry represents. | 'DIRECT_CONVERTER', 'DIRECT_READER', 'FILE_READER', 'HASH_CONVERTER', 'HASH_READER', 'ROUNDROBIN_CONVERTER', 'EXPORT_READER', 'EXTERNAL_READER', 'EXTERNAL_WRITER', 'PARALLEL_COPY_READER', 'REJECT_WRITER', 'WRITER'
status | nvarchar(32) | Status of the DMS worker. | Information not available.
bytes_per_sec | bigint | Read or write throughput in the last second. | Greater than or equal to 0.
bytes_processed | bigint | Total bytes processed by this worker. | Greater than or equal to 0.
rows_processed | bigint | Number of rows read or written for this worker. | Greater than or equal to 0.
start_time | datetime | Time at which execution of this worker started. | Greater than or equal to start time of the query step this worker belongs to. See sys.dm_pdw_request_steps (SQL Server PDW).
end_time | datetime | Time at which execution ended, failed, or was cancelled. | NULL for ongoing or queued workers. Otherwise, greater than start_time.
total_elapsed_time | int | Total time spent in execution, in milliseconds. | Greater than or equal to 0. Total time elapsed since system start or restart. If total_elapsed_time exceeds the maximum value for an integer (24.8 days in milliseconds), it will cause materialization failure due to overflow. The maximum value in milliseconds is equivalent to 24.8 days.
cpu_time | bigint | CPU time consumed by this worker, in milliseconds. | Greater than or equal to 0.
query_time | int | Period of time before SQL starts returning rows to the thread, in milliseconds. | Greater than or equal to 0.
buffers_available | int | Number of unused buffers. |
sql_spid | int | Session id on the SQL Server instance performing the work for this DMS worker. |
dms_cpid | int | Process ID of the actual thread running. |
error_id | nvarchar(36) | Unique identifier of the error that occurred during execution of this worker, if any. | See error_id in sys.dm_pdw_errors (SQL Server PDW)
source_info | nvarchar(4000) | For a reader, specification of the source tables and columns. |
destination_info | nvarchar(4000) | For a writer, specification of the destination tables. |

For information about the maximum rows retained by this view, see Maximum System View Values.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_errors (SQL Server PDW)

[image]

sys.dm_pdw_errors (SQL Server PDW)
See Also Send Feedback

Holds information about all errors encountered during execution of a request or query.

Column Name | Data Type | Description | Range
error_id | nvarchar(36) | Key for this view. Unique numeric id associated with the error. | Unique across all query errors in the system.
source | nvarchar(64) | Information not available. | Information not available.
type | nvarchar(4000) | Type of error that occurred. | Information not available.
create_time | datetime | Time at which the error occurred. | Smaller or equal to current time.
pwd_node_id | int | Identifier of the specific node involved, if any. For additional information on node ids, see sys.dm_pdw_nodes (SQL Server PDW). |
session_id | nvarchar(32) | Identifier of the session involved, if any. For additional information on session ids, see sys.dm_pdw_exec_sessions (SQL Server PDW). |
request_id | nvarchar(32) | Identifier of the request involved, if any. For additional information on request ids, see sys.dm_pdw_exec_requests (SQL Server PDW). |
spid | int | spid of the SQL Server session involved, if any. |
thread_id | int | Information not available. |
details | nvarchar(4000) | Holds the full error text description. |

For information about the maximum rows retained by this view, see the Maximum System View Values section in the Minimum and Maximum Values (SQL Server PDW) topic.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_exec_connections (SQL Server PDW)

[image]

sys.dm_pdw_exec_connections (SQL Server PDW)
See Also Send Feedback

Returns information about the connections established to this instance of SQL Server PDW and the details of each connection.

Column name | Data type | Description
session_id | int | Identifies the session associated with this connection. Use SESSION_ID() to return the session_id of the current connection.
connect_time | datetime | Timestamp when connection was established. Is not nullable.
encrypt_option | nvarchar(40) | Indicates TRUE (connection is encrypted) or FALSE (connection is not enctypred).
auth_scheme | nvarchar(40) | Specifies SQL Server/Windows Authentication scheme used with this connection. Is not nullable.
client_id | varchar(48) | IP address of the client connecting to this server. Is nullable.
sql_spid | int | The server process ID of the connection. Use @@SPID to return the sql_spid of the current connection.For most purposed, use the session_id instead.

Permissions

Requires VIEW SERVER STATE permission on the server.

Relationship Cardinalities

dm_pdw_exec_sessions.session_id | dm_pdw_exec_connections.session_id | One-to-one
dm_pdw_exec_requests.connection_id | dm_pdw_exec_connections.connection_id | Many to one

Examples

Typical query to gather information about a queries own connection.

Copy image Copy Code
SELECT
    c.session_id, c.encrypt_option,
    c.auth_scheme, s.client_id, s.login_name,
    s.status, s.query_count
FROM sys.dm_pdw_exec_connections AS c
JOIN sys.dm_pdw_exec_sessions AS s
    ON c.session_id = s.session_id
WHERE c.session_id = SESSION_ID();

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_exec_requests (SQL Server PDW)

[image]

sys.dm_pdw_exec_requests (SQL Server PDW)
See Also Send Feedback

Holds information about all requests currently or recently active in SQL Server PDW. It lists one row per request/query.

Column Name | Data Type | Description | Range
request_id | nvarchar(32) | Key for this view. Unique numeric id associated with the request. | Unique across all requests in the system.
session_id | nvarchar(32) | Unique numeric id associated with the session in which this query was run. See sys.dm_pdw_exec_sessions (SQL Server PDW). |
status | nvarchar(32) | Current status of the request. | 'Pending', 'Authorizing', 'AcquireSystemResources', 'Initializing', 'Plan', 'Parsing', 'AquireResources', 'Running', 'Cancelling', 'Complete', 'Failed', 'Cancelled'.
submit_time | datetime | Time at which the request was submitted for execution. | Valid datetime smaller or equal to the current time and start_time.
start_time | datetime | Time at which the request execution was started. | 0 for queued requests; otherwise, valid datetime smaller or equal to current time.
end_compile_time | datetime | Time at which the engine completed compiling the request. | 0 for requests that have not been compiled yet; otherwise a valid datetime greater than start_time and less than or equal to the current time.
end_time | datetime | Time at which the request execution completed, failed, or was cancelled. | Null for queued or active requests; otherwise, a valid datetime smaller or equal to current time.
total_elapsed_time | int | Time elapsed in execution since the request was started, in milliseconds. | Between 0 and the difference between start_time and end_time. If total_elapsed_time exceeds the maximum value for an integer, total_elapsed_time will continue to be the maximum value. This condition will generate the warning “The maximum value has been exceeded.” The maximum value in milliseconds is equivalent to 24.8 days.
label | nvarchar(255) | Optional label string associated with some SELECT query statements. | Any string containing 'a-z','A-Z','0-9','_'.
error_id | nvarchar(36) | Unique id of the error associated with the request, if any. | See sys.dm_pdw_errors (SQL Server PDW); set to NULL if no error occurred.
database_id | int | Identifier of database used by explicit context (e.g., USE DB_X). | See id in sys.databases (SQL Server PDW).
command | nvarchar(4000) | Holds the full text of the request as submitted by the user. | Any valid query or request text. Queries that are longer than 4000 bytes are truncated.
resource_class | nvarchar(20) | The resource class for this request. See related concurrency_slots_used in sys.dm_pdw_resource_waits (SQL Server PDW). | SmallRC MediumRC LargeRC XLargeRC

For information about the maximum rows retained by this view, see the Maximum System View Values section in the Minimum and Maximum Values (SQL Server PDW) topic.

Permissions

Requires VIEW SERVER STATE permission.

Security

sys.dm_pdw_exec_requests does not filter query results according to database-specific permissions. Logins with VIEW SERVER STATE permission can obtain results query results for all databases

Caution note Caution

An attacker can use sys.dm_pdw_exec_requests to retrieve information about specific database objects by simply having VIEW SERVER STATE permission and by not having a database-specific permission.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_exec_sessions (SQL Server PDW)

[image]

sys.dm_pdw_exec_sessions (SQL Server PDW)
See Also Send Feedback

Holds information about all sessions currently or recently open on the appliance. It lists one row per session.

Column Name

Data Type

Description

Range

session_id

nvarchar(32)

The id of the current query or the last query run (if the session is TERMINATED and the query was executing at time of termination). Key for this view.

Unique across all sessions in the system.

status

nvarchar(10)

For current sessions, identifies whether the session is currently active or idle. For past sessions, the session status may show closed or killed (if the session was forcibly closed).

'ACTIVE', 'CLOSED', 'IDLE', 'TERMINATED'

request_id

nvarchar(32)

The id of the current query or last query run.

Unique across all requests in the system. Null if none has been run.

security_id

varbinary(85)

Security ID of the principal running the session.

login_name

nvarchar(128)

The login name of the principal running the session.

Any string conforming to the user naming conventions.

login_time

datetime

Date and time at which the user logged in and this session was created.

Valid datetime before current time.

query_count

int

Captures the number of queries/requests this session has run since creation.

Greater than or equal to 0.

is_transactional

bit

Captures whether a session is currently within a transaction or not.

0 for auto-commit, 1 for transactional.

client_id

nvarchar(255)

Captures client information for the session.

Any valid string.

app_name

nvarchar(255)

Captures application name information optionally set as part of the connection process.

Any valid string.

sql_spid

int

The id number of the SPID. Use the session_id this session. Use the sql_spid column to join to sys.dm_pdw_nodes_exec_sessions.

Caution note Caution

This column contains closed SPIDs.

For information about the maximum rows retained by this view, see the Maximum System View Values section in the Minimum and Maximum Values (SQL Server PDW) topic.

Permissions

Requires the VIEW SERVER STATE permission.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_network_credentials (SQL Server PDW)

[image]

sys.dm_pdw_network_credentials (SQL Server PDW)
See Also Send Feedback

Returns a list of all network credentials stored in the SQL Server PDW appliance for all target servers. Results are listed for the Control node, and every Compute node.

Column Name | Data Type | Description
pdw_node_id | int | Unique numeric id associated with the node.
target_server_name | nvarchar(32) | IP address of the target server that SQL Server PDW will access by using the username and password credentials.
username | nvarchar(32) | Username for which the password is stored.
last_modified | datetime | Datetime of the last operation that modified the credential.

Permissions

Requires VIEW SERVER STATE.

General Remarks

The key for this dynamic management view is pdw_node_id plus target_server_name.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_nodes (SQL Server PDW)

[image]

sys.dm_pdw_nodes (SQL Server PDW)
See Also Send Feedback

Holds information about all of the nodes in Analytics Platform System. It lists one row per node in the appliance.

Column Name | Data Type | Description | Range
pdw_node_id | int | Unique numeric id associated with the node. Key for this view. | Unique across the appliance, regardless of type.
type | nvarchar(32) | Type of the node. | 'COMPUTE', 'CONTROL', 'MANAGEMENT'
name | nvarchar(32) | Logical name of the node. | Any string of appropriate length.
address | nvarchar(32) | IP address of this node. | In the format of [0-255].[0-255].[0-255].[0-255].
is_passive | int | Indicates whether the virtual machine running the node is running on the assigned server or has failed over to the spare server. | 0 – node VM is running on the original server. 1 – node VM is running on the spare server.
region | nvarchar(32) | The region where the node is running. | 'PDW', 'HDINSIGHT'

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_nodes_database_encryption_keys (SQL Server PDW)

[image]

sys.dm_pdw_nodes_database_encryption_keys (SQL Server PDW)
See Also Send Feedback

Returns information about the encryption state of a database and its associated database encryption keys. sys.dm_pdw_nodes_database_encryption_keys provides this information for each node. For more information about database encryption, see Transparent Data Encryption (SQL Server PDW).

Column Name | Data Type | Description
database_id | int | ID of the physical database on each node.
encryption_state | int | Indicates whether the database on this node is encrypted or not encrypted. 0 = No database encryption key present, no encryption 1 = Unencrypted 2 = Encryption in progress 3 = Encrypted 4 = Key change in progress 5 = Decryption in progress 6 = Protection change in progress (The certificate that is encrypting the database encryption key is being changed.)
create_date | datetime | Displays the date the encryption key was created.
regenerate_date | datetime | Displays the date the encryption key was regenerated.
modify_date | datetime | Displays the date the encryption key was modified.
set_date | datetime | Displays the date the encryption key was applied to the database.
opened_date | datetime | Shows when the database key was last opened.
key_algorithm | varchar(?) | Displays the algorithm that is used for the key.
key_length | int | Displays the length of the key.
encryptor_thumbprint | varbin | Shows the thumbprint of the encryptor.
percent_complete | real | Percent complete of the database encryption state change. This will be 0 if there is no state change.
node_id | int | Unique numeric id associated with the node.

Permissions

Requires the VIEW SERVER STATE permission on the server.

The following example joins sys.dm_pdw_nodes_database_encryption_keys to other system tables to indicate the encryption state for each node of the TDE protected databases.

Copy image Copy Code
SELECT D.database_id AS DBIDinMaster, D.name AS UserDatabaseName,
PD.pdw_node_id AS NodeID, DM.physical_name AS PhysDBName,
keys.encryption_state
FROM sys.dm_pdw_nodes_database_encryption_keys AS keys
JOIN sys.pdw_nodes_pdw_physical_databases AS PD
    ON keys.database_id = PD.database_id AND keys.pdw_node_id = PD.pdw_node_id
JOIN sys.pdw_database_mappings AS DM
    ON DM.physical_name = PD.physical_name
JOIN sys.databases AS D
    ON D.database_id = DM.database_id
ORDER BY D.database_id, PD.pdw_node_ID;

See Also

Reference

CREATE DATABASE ENCRYPTION KEY (SQL Server PDW)
ALTER DATABASE ENCRYPTION KEY (SQL Server PDW)
DROP DATABASE ENCRYPTION KEY (SQL Server PDW)
sp_pdw_database_encryption_regenerate_system_keys (SQL Server PDW)

Concepts

Transparent Data Encryption (SQL Server PDW)


sys.dm_pdw_node_status (SQL Server PDW)

[image]

sys.dm_pdw_node_status (SQL Server PDW)
See Also Send Feedback

Holds additional information (over sys.dm_pdw_nodes (SQL Server PDW)) about the performance and status of all appliance nodes. It lists one row per node in the appliance.

Column Name | Data Type | Description | Range
pdw_node_id | int | Unique numeric id associated with the node. Key for this view. | Unique across the appliance, regardless of type.
process_id | int | Information not available. |
process_name | nvarchar(255) | Information not available. |
allocated_memory | bigint | Total allocated memory on this node. |
available_memory | bigint | Total available memory on this node. |
process_cpu_usage | bigint | Total process CPU usage, in ticks. |
total_cpu_usage | bigint | Total CPU usage, in ticks. |
thread_count | bigint | Total number of threads in use on this node. |
handle_count | bigint | Total number of handles in use on this node. |
total_elapsed_time | bigint | Total time elapsed since system start or restart. | Total time elapsed since system start or restart. If total_elapsed_time exceeds the maximum value for an integer (24.8 days in milliseconds), it will cause materialization failure due to overflow. The maximum value in milliseconds is equivalent to 24.8 days.
is_available | bit | Flag indicating whether this node is available. |
sent_time | datetime | Last time a network package was sent by this node. |
received_time | datetime | Last time a network package was received by this node. |
error_id | nvarchar(36) | Unique identifier of the last error that occurred on this node. |

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_os_event_logs (SQL Server PDW)

[image]

sys.dm_pdw_os_event_logs (SQL Server PDW)
See Also Send Feedback

Holds information regarding the different Windows Event logs on the different nodes.

Column Name | Data Type | Description | Range
pdw_node_id | int | Appliance node this log is from. pdw_node_id and log_name form the key for this view. |
log_name | nvarchar(255) | Windows event log name. pdw_node_id and log_name form the key for this view. |
log_source | nvarchar(255) | Windows event log source name. |
event_id | int | ID of the event. Not unique. |
event_type | nvarchar(255) | Type of the event, identifying severity. | 'Information', 'Warning', 'Error'
event_message | nvarchar(4000) | Details of the event. |
generate_time | datetime | Time the event was created. |
write_time | datetime | Time the event was actually written to the log. |

For information about the maximum rows retained by this view, see the Maximum System View Values section in the Minimum and Maximum Values (SQL Server PDW) topic.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_os_performance_counters (SQL Server PDW)

[image]

sys.dm_pdw_os_performance_counters (SQL Server PDW)
See Also Send Feedback

Contains information about Windows performance counters for the nodes in SQL Server PDW.

Column Name | Data Type | Description | Range
pdw_node_id | int | The ID of the node that contains the counter. pdw_node_id and counter_name form the key for this view. | See node_id in sys.sys.dm_pdw_nodes (SQL Server PDW).
counter_name | nvarchar(255) | Name of Windows performance counter. |
counter_category | nvarchar(255) | Name of Windows performance counter category. |
instance_name | nvarchar(255) | Name of the specific instance of the counter. |
counter_value | Decimal(38,10) | Current value of the counter. |
last_update_time | Datetime2(3) | Timestamp of last time the value was updated. |

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_os_threads (SQL Server PDW)

[image]

sys.dm_pdw_os_threads (SQL Server PDW)
See Also Send Feedback

Column Name | Data Type | Description | Range
pdw_node_id | int | The ID of the affected node. pdw_node_id and thread_id form the key for this view. | See node_id in sys.sys.dm_pdw_nodes (SQL Server PDW).
thread_id | int | pdw_node_id and thread_id form the key for this view. |
process_id | int |  |
name | nvarchar(255) |  |
priority | int |  |
start_time | datetime |  |
state | nvarchar(32) |  |
wait_reason | nvarchar(32) |  |
total_processor_elapsed_time | bigint | Total kernel time used by the thread. |
total_user_elapsed_time | bigint | Total user time used by the thread |

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_query_stats_xe (SQL Server PDW)

[image]

sys.dm_pdw_query_stats_xe (SQL Server PDW)
See Also Send Feedback

This DMV is deprecated and will be removed in a future release. In this release, it returns 0 rows.

Column Name | Data Type | Description | Range
event | nvarchar(60) | Key for this view. |
event_id | nvarchar(36) |  |
create_time | datetime |  |
session_id | int | The id for the session. | See session_id in sys.dm_pdw_exec_sessions (SQL Server PDW).
cpu | int |  |
reads | int | Number of logical reads since the start of the event. |
writes | int | Number of logical writes since the start of the event. |
sql_text | nvarchar(4000) |  |
client_app_name | nvarchar(255) |  |
tsql_stack | nvarchar(255) |  |
pdw_node_id | int | Node on which this Xevent instance is running.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_query_stats_xe_file (SQL Server PDW)

[image]

sys.dm_pdw_query_stats_xe_file (SQL Server PDW)
See Also Send Feedback

This DMV is deprecated and will be removed in a future release. In this release, it returns 0 rows.

Column Name | Data Type | Description | Range
event | nvarchar(60) | Key for this view. |
data | xml |  |
pdw_node_id | int | Node on which this Xevent instance is running. |

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_request_steps (SQL Server PDW)

[image]

sys.dm_pdw_request_steps (SQL Server PDW)
See Also Send Feedback

Holds information about all steps that compose a given request or query in SQL Server PDW. It lists one row per query step.

Column Name

Data Type

Description

Range

request_id

nvarchar(32)

request_id and step_index make up the key for this view.

Unique numeric id associated with the request.

See request_id in sys.dm_pdw_exec_requests (SQL Server PDW).

step_index

int

request_id and step_index make up the key for this view.

The position of this step in the sequence of steps that make up the request.

0 to (n-1) for a request with n steps.

operation_type

nvarchar(32)

Type of operation represented by this step.

DMS query plan operations: 'ReturnOperation', 'PartitionMoveOperation', 'MoveOperation', 'BroadcastMoveOperation', 'ShuffleMoveOperation', 'TrimMoveOperation', 'CopyOperation', 'DistributeReplicatedTableMoveOperation'

SQL query plan operations: 'OnOperation', 'RemoteOperation'

Other query plan operations: 'MetaDataCreateOperation', 'RandomIDOperation'

External operations for reads: 'HadoopShuffleOperation', 'HadoopRoundRobinOperation', 'HadoopBroadcastOperation'

External operations for MapReduce: 'HadoopJobOperation', 'HdfsDeleteOperation'

External operations for writes: 'ExternalExportDistributedOperation', 'ExternalExportReplicatedOperation', 'ExternalExportControlOperation'

For more information, see Understanding Query Plans (SQL Server PDW).

distribution_type

nvarchar(32)

Type of distribution this step will undergo.

'AllNodes', 'AllDistributions', 'AllComputeNodes', 'ComputeNode', 'Distribution', 'SubsetNodes', 'SubsetDistributions', 'Unspecified'

location_type

nvarchar(32)

Where the step is running.

'Compute', 'Control'

status

nvarchar(32)

Status of this step.

'Pending', 'Running', 'Complete', 'Failed', 'UndoFailed', 'PendingCancel', 'Cancelled', 'Undone', 'Aborted'.

error_id

nvarchar(36)

Unique id of the error associated with this step, if any.

See error_id of sys.dm_pdw_request_errors (SQL Server PDW). NULL if no error occurred.

start_time

datetime

Time at which the step started execution.

Smaller or equal to current time and larger or equal to end_compile_time of the query to which this step belongs. For more information on queries, see sys.dm_pdw_exec_requests (SQL Server PDW).

end_time

datetime

Time at which this step completed execution, was cancelled, or failed.

Smaller or equal to current time and larger or equal to start_time. Set to NULL for steps currently in execution or queued.

total_elapsed_time

int

Total amount of time the query step has been running, in milliseconds.

Between 0 and the difference between end_time and start_time. 0 for queued steps.

If total_elapsed_time exceeds the maximum value for an integer, total_elapsed_time will continue to be the maximum value. This condition will generate the warning “The maximum value has been exceeded.”

The maximum value in milliseconds is equivalent to 24.8 days.

row_count

bigint

Total number of rows changed or returned by this request.

0 for steps that did not change or return data. Otherwise, number of rows affected.

command

nvarchar(4000)

Holds the full text of the command of this step.

Any valid request string for a step. Truncated if longer than 4000 characters.

For information about the maximum rows retained by this view, see the Maximum System View Values section in the Minimum and Maximum Values (SQL Server PDW) topic.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_resource_waits (SQL Server PDW)

[image]

sys.dm_pdw_resource_waits (SQL Server PDW)
See Also Send Feedback

Displays wait information for all resource types in SQL Server PDW.

Column Name

Data Type

Description

Range

wait_id

bigint

Position of the request in the waiting list.

0-based ordinal. This is not unique across all wait entries.

session_id

nvarchar(32)

ID of the session in which the wait state occurred.

See session_id in sys.dm_pdw_exec_sessions (SQL Server PDW).

type

nvarchar(255)

Type of wait this entry represents.

Possible values:

  • Connection

  • Local Queries Concurrency

  • Distributed Queries Concurrency

  • DMS Concurrency

  • Backup Concurrency

object_type

nvarchar(255)

Type of object that is affected by the wait.

Possible values:

  1. OBJECT

  2. DATABASE

  3. SYSTEM

  4. SCHEMA

  5. APPLICATION

object_name

nvarchar(386)

Name or GUID of the specified object that was affected by the wait.

Tables and views are displayed with three-part names.

Indexes and statistics are displayed with four-part names.

Names, principals, and databases are string names.

request_id

nvarchar(32)

ID of the request on which the wait state occurred.

QID identifier of the request.

GUID identifier for load requests.

request_time

datetime

Time at which the lock or resource was requested.

acquire_time

datetime

Time at which the lock or resource was acquired.

state

nvarchar(50)

State of the wait state.

Information not available.

priority

int

Priority of the waiting item.

Information not available.

concurrency_slots_used

int

Number of concurrency slots (32 max) reserved for this request.

1 – for SmallRC

3 – for MediumRC

7 for LargeRC

22 – for XLargeRC

resource_class

nvarchar(20)

The resource class for this request.

SmallRC

MediumRC

LargeRC

XLargeRC

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_sql_requests (SQL Server PDW)

[image]

sys.dm_pdw_sql_requests (SQL Server PDW)
See Also Send Feedback

Holds information about all SQL Server query distributions as part of a SQL step in the query.

Column Name

Data Type

Description

Range

request_id

nvarchar(32)

Unique identifier of the query to which this SQL query distribution belongs.

request_id, step_index, and distribution_id form the key for this view.

See request_id in sys.dm_pdw_exec_requests (SQL Server PDW)

step_index

int

Index of the query step this distribution is part of.

request_id, step_index, and distribution_id form the key for this view.

See step_index in sys.dm_pdw_request_steps (SQL Server PDW)

pwd_node_id

int

Unique identifier of the node on which this query distribution is run.

See node_id in sys.dm_pdw_nodes (SQL Server PDW).

distribution_id

int

Unique identifier of the distribution on which this query distribution is run.

request_id, step_index, and distribution_id form the key for this view.

See distribution_id in sys.pdw_distributions (SQL Server PDW). Set to -1 for requests that run at the node scope, not the distribution scope.

status

nvarchar(32)

Current status of the query distribution.

'ACTIVE','CANCELLED','COMPLETED','FAILED','QUEUED'

error_id

nvarchar(36)

Unique identifier of the error associated with this query distribution, if any.

See error_id in sys.dm_pdw_errors (SQL Server PDW). Set to NULL if no error occurred.

start_time

datetime

Time at which query distribution started execution.

Smaller or equal to current time and greater or equal to start_time of the query step this query distribution belongs to

end_time

datetime

Time at which this query distribution completed execution, was cancelled, or failed.

Greater or equal to start time, or set to NULL if the query distribution is ongoing or queued.

total_elapsed_time

int

Represents the time the query distribution has been running, in milliseconds.

Greater or equal to 0. Equal to the delta of start_time and end_time for completed, failed, or cancelled query distributions.

If total_elapsed_time exceeds the maximum value for an integer, total_elapsed_time will continue to be the maximum value. This condition will generate the warning “The maximum value has been exceeded.”

The maximum value in milliseconds is equivalent to 24.8 days.

row_count

bigint

Number of rows changed or read by this query distribution.

0 for query distributions that do not change or return data.

spid

int

Session id on the SQL Server instance running the query distribution.

command

nvarchar(4000)

Full text of command for this query distribution.

Any valid query or request string.

For information about the maximum rows retained by this view, see the Maximum System View Values section in the Minimum and Maximum Values (SQL Server PDW) topic.

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_sys_info (SQL Server PDW)

[image]

sys.dm_pdw_sys_info (SQL Server PDW)
See Also Send Feedback

Provides a set of appliance-level counters that reflect overall activity on the appliance.

Column Name | Data Type | Description | Range
total_sessions | int | Number of sessions currently in the system. | 0 to max_active_sessions (see below).
idle_sessions | int | Number of sessions currently idle. |
active_requests | int | Number of active requests currently running. |
queued_requests | int | Number of currently queued requests. |
active_loads | int | Number of loads currently running in the system. |
queued_loads | int | Number of queued loads waiting for execution. |
active_backups | int | Number of backups currently running. |
active_restores | int | Number of backup restores currently running. |

See Also

Reference

System Views (SQL Server PDW)


sys.dm_pdw_wait_stats (SQL Server PDW)

[image]

sys.dm_pdw_wait_stats (SQL Server PDW)
See Also Send Feedback

Holds information related to the SQL Server OS state related to instances running on the different nodes. For a list of waits types and their description, see sys.dm_os_wait_stats.

Column Name | Data Type | Description | Range
pdw_node_id | int | ID of the node this entry refers to. |
wait_name | nvarchar(255) | Name of the wait type. |
max_wait_time | bigint | Maximum wait time of this wait type. |
request_count | bigint | Number of waits of this wait type outstanding. |
signal_time | bigint | Difference between the time that the waiting thread was signaled and when it started running. |
completed_count | bigint | Total number of waits of this type completed since the last server restart. |
wait_time | bigint | Total wait time for this wait type in millisecons. Inclusive of signal_time. |

See Also

Reference

sys.dm_pdw_waits (SQL Server PDW)
System Views (SQL Server PDW)


sys.dm_pdw_waits (SQL Server PDW)

[image]

sys.dm_pdw_waits (SQL Server PDW)
See Also Send Feedback

Holds information about all wait states encountered during execution of a request or query, including locks, waits on transmission queues, and so on.

Column Name | Data Type | Description | Range
wait_id | bigint | Unique numeric id associated with the wait state. Key for this view. | Unique across all waits in the system.
session_id | nvarchar(32) | ID of the session on which the wait state occurred. | See session_id in sys.dm_pdw_exec_sessions (SQL Server PDW).
type | nvarchar(255) | Type of wait this entry represents. | Information not available.
object_type | nvarchar(255) | Type of object that is affected by the wait. | Information not available.
object_name | nvarchar(386) | Name or GUID of the specified object that was affected by the wait. |
request_id | nvarchar(32) | ID of the request on which the wait state occurred. | See request_id in sys.dm_pdw_exec_requests (SQL Server PDW).
request_time | datetime | Time at which the wait state was requested. |
acquire_time | datetime | Time at which the lock or resource was acquired. |
state | nvarchar(50) | State of the wait state. | Information not available.
priority | int | Priority of the waiting item. | Information not available.

See Also

Reference

sys.dm_pdw_wait_stats (SQL Server PDW)
System Views (SQL Server PDW)


sys.dm_pdw_lock_waits (SQL Server PDW)

[image]

sys.dm_pdw_lock_waits (SQL Server PDW)
See Also Send Feedback

Holds information about the requests that are waiting for locks.

Column Name

Data Type

Description

Range

wait_id

bigint

Position of the request in the waiting list.

0-based ordinal. This is not unique across all wait entries.

session_id

nvarchar(32)

ID of the session in which the wait state occurred.

See session_id in sys.dm_pdw_exec_sessions (SQL Server PDW).

type

nvarchar(255)

Type of wait this entry represents.

Possible values:

  • Shared

  • SharedUpdate

  • ExclusiveUpdate

  • Exclusive

Shared

object_type

nvarchar(255)

Type of object that is affected by the wait.

Possible values:

  1. OBJECT

  2. DATABASE

  3. SYSTEM

  4. SCHEMA

  5. APPLICATION

object_name

nvarchar(386)

Name or GUID of the specified object that was affected by the wait.

Tables and views are displayed with three-part names.

Indexes and statistics are displayed with four-part names.

Names, principals, and databases are string names.

request_id

nvarchar(32)

ID of the request on which the wait state occurred.

ID of the request.

This is a GUID for load requests.

request_time

datetime

Time at which the lock or resource was requested.

acquire_time

datetime

Time at which the lock or resource was acquired.

state

nvarchar(50)

State of the wait state.

Information not available.

priority

int

Priority of the waiting item.

Information not available.

See Also

Reference

System Views (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
Locking Behavior (SQL Server PDW)


SQL Server Catalog Views (SQL Server PDW)

[image]

SQL Server Catalog Views (SQL Server PDW)
See Also Send Feedback

This topic lists the SQL Server catalog views available in SQL Server PDW.

SQL Server Catalog Views

These views have a few PDW-specific modifications, such as range differences or possible values, that differ from SQL Server. For more details, click on the links to see the SQL Server PDW documentation.

Note Note

The data types nvarchar(max), uniqueidentifier are supported in SQL Server catalog views only and cannot be used with or for user data. The data types sysname and sql_variant are supported in SQL Server catalog views, in variables, and with the CAST function, but cannot be used in user tables.

sys.all_columns (SQL Server PDW)

sys.all_objects (SQL Server PDW)

sys.all_sql_modules (SQL Server PDW)

sys.all_views (SQL Server PDW)

sys.assemblies (SQL Server PDW)

sys.assembly_modules (SQL Server PDW)

sys.assembly_types (SQL Server PDW)

sys.check_constraints (SQL Server PDW)

sys.columns (SQL Server PDW)

sys.computed_columns (SQL Server PDW)

sys.data_spaces (SQL Server PDW)

sys.database_files (SQL Server PDW)

sys.database_permissions (SQL Server PDW)

sys.database_principals (SQL Server PDW)

sys.database_role_members (SQL Server PDW)

sys.databases (SQL Server PDW)

sys.default_constraints (SQL Server PDW)

sys.extended_properties (SQL Server PDW)

sys.filegroups (SQL Server PDW)

sys.foreign_keys (SQL Server PDW)

sys.foreign_key_columns (SQL Server PDW)

sys.identity_columns (SQL Server PDW)

sys.filegroups (SQL Server PDW)

sys.foreign_keys (SQL Server PDW)

sys.foreign_key_columns (SQL Server PDW)

sys.identity_columns (SQL Server PDW)

sys.index_columns (SQL Server PDW)

sys.indexes (SQL Server PDW)

sys.key_constraints (SQL Server PDW)

sys.master_files (SQL Server PDW)

sys.numbered_procedures (SQL Server PDW)

sys.objects (SQL Server PDW)

sys.partition_functions (SQL Server PDW)

sys.partition_parameters (SQL Server PDW)

sys.partition_range_values (SQL Server PDW)

sys.partition_schemes (SQL Server PDW)

sys.partitions (SQL Server PDW)

sys.schemas (SQL Server PDW)

sys.securable_classes (SQL Server PDW)

sys.server_role_members (SQL Server PDW)

sys.server_permissions (SQL Server PDW)

sys.server_principals (SQL Server PDW)

sys.sql_expression_dependencies (SQL Server PDW)

sys.sql_logins (SQL Server PDW)

sys.sql_modules (SQL Server PDW)

sys.stats (SQL Server PDW)

sys.stats_columns (SQL Server PDW)

sys.system_columns (SQL Server PDW)

sys.system_objects (SQL Server PDW)

sys.system_sql_modules (SQL Server PDW)

sys.system_views (SQL Server PDW)

sys.tables (SQL Server PDW)

sys.types (SQL Server PDW)

sys.views (SQL Server PDW)

See Also

Reference

System Views (SQL Server PDW)


sys.all_columns (SQL Server PDW)

[image]

sys.all_columns (SQL Server PDW)
See Also Send Feedback

Shows columns for both user-defined and system objects.

Column Name | Data Type | Description | Range
object_id | int | ID of the object to which this column belongs. |
name | sysname | Name of the column. Unique in object. |
column_id | int | ID of the column. Unique in object. |
system_type_id | tinyint | ID of the system type of the column. |
user_type_id | int | ID of the type of the column as defined by the user. |
max_length | smallint | Maximum length (in bytes) of the column. |
precision | tinyint | Precision of the column if numeric-based; otherwise, 0. |
scale | tinyint | Scale of column if numeric-based; otherwise, 0. |
collation_name | sysname | Name of the collation of the column if character-based; otherwise NULL. |
is_nullable | bit | 1 = Column is nullable. |
is_ansi_padded | bit | 1 = Column uses ANSI_PADDING ON behavior if character, binary, or variant. | Always 0.
is_rowguidcol | bit | 1 = Column is a declared ROWGUIDCOL. | Always 0.
is_identity | bit | 1 = Column has identity values. | Always 0.
is_computed | bit | 1 = Column is a computed column. | Always 0.
is_filestream | bit | 1 = Column is a FILESTREAM column. | Always 0.
is_replicated | bit | 1 = Column is replicated. | Always 0.
is_non_sql_subscribed | bit | 1 = Column has a non-SQL subscriber. | Always 0.
is_merge_published | bit | 1 = Column is merge-published. | Always 0.
is_dts_replicated | bit | 1 = Column is replicated by using SSIS. | Always 0.
is_xml_document | bit | 1 = Content is a complete XML document. | Always 0.
xml_collection_id | int | 0 = No XML schema collection. | Always 0.
default_object_id | int | ID of the default object, 0 = No default. | Always 0.
rule_object_id | int | ID of the stand-alone rule bound to the column; 0 = No stand-alone rule. | Always 0.
is_sparse | bit | 1 = Column is a sparse column. | Always 0.
is_column_set | bit | 1 = Column is a column set. | Always 0.
distribution_ordinal | tinyint | Ordinal (1-based) within set of distribution. 0 = Not a distribution column. | 1 = This is the distribution column. 0 = This is not the distribution column.

See Also

Reference

sys.columns (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.all_objects (SQL Server PDW)

[image]

sys.all_objects (SQL Server PDW)
See Also Send Feedback

Returns user-defined and system objects, including tables and views.

Column Name | Data Type | Description | Range
name | sysname | Object name. |
object_id | int | Object identification number. Is unique within a database. |
principal_id | int | ID of the individual owner, if different from the schema owner. | See principal_id in sys.database_principals (SQL Server PDW).
schema_id | int | ID of the schema that the object is contained in. Schema-scoped system objects are always contained in the sys or INFORMATION_SCHEMA schemas. |
parent_object_id | int | ID of the object to which this object belongs. 0 = Not a child object. |
type | char(2) | Object type: FN = SQL scalar function IT = Internal table S = System base table U = Table (user-defined) V = View | 'FN', 'IT', 'S', 'U', 'V'
type_desc | nvarchar(60) | SQL_SCALAR_FUNCTION INTERNAL_TABLE SYSTEM_TABLE USER_TABLE VIEW | 'SQL_SCALAR_FUNCTION', 'INTERNAL_TABLE', 'SYSTEM_TABLE', 'USER_TABLE, VIEW'
create_date | datetime | Date the object was created. |
modify_date | datetime | Date the object was last modified by using an ALTER statement. If the object is a table or a view, modify_date also changes when a clustered index on the table or view is created or altered. |
is_ms_shipped | bit | Object is created by an internal SQL Server component. |
is_published | bit | Object is published. | Always 0.
is_schema_published | bit | Only the schema of the object is published. | Always 0.

See Also

Reference

sys.objects (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.all_sql_modules (SQL Server PDW)

[image]

sys.all_sql_modules (SQL Server PDW)
See Also Send Feedback

Returns SQL Server modules, including user-defined and system objects.

Column Name | Data Type | Description | Range
object_id | int | ID of the object of the containing object. Is unique within a database. |
definition | nvarchar(max) | SQL text that defines this module. |
uses_ansi_nulls | bit | Module was created with SET ANSI_NULLS ON. Will always be = 0 for rules and defaults. |
uses_quoted_identifier | bit | Module was created with SET QUOTED_IDENTIFIER ON. |
is_schema_bound | bit | Module was created with SCHEMABINDING option. |
uses_database_collation | bit |  | Always 0.
is_recompiled | bit | Procedure was created WITH RECOMPILE option. | Always 0.
null_on_null_input | bit | Module was declared to produce a NULL output on any NULL input. | Always 0.
execute_as_principal_id | int |  | Always NULL.

See Also

Reference

sys.sql_modules (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.all_views (SQL Server PDW)

[image]

sys.all_views (SQL Server PDW)
See Also Send Feedback

Shows user defined and system objects.

Column Name | Data Type | Description | Range
<inherited columns from sys.objects> |  | See sys.objects (SQL Server PDW) |
is_replicated | bit | 1 = View is replicated. | Always 0.
has_replication_filter | bit | 1 = View has a replication filter. | Always 0.
has_opaque_metadata | bit | 1 = VIEW_METADATA option specified for view. | Always 0.
has_unchecked_assembly_data | bit | Information not available. | Always 0, no CLR support.
with_check_option | bit | 1 = WITH CHECK OPTION was specified in the view definition. | Always 0.
is_date_correlation_view | bit | 1 = View was created automatically by the system to store correlation information between datetime columns. | Always 0.

See Also

Reference

System Views (SQL Server PDW)


sys.assemblies (SQL Server PDW)

[image]

sys.assemblies (SQL Server PDW)
See Also Send Feedback

Returns a row for each assembly.

Column name | Data type | Description
name | sysname | Name of the assembly. Is unique within the database.
principal_id | int | ID of the principal that owns this assembly.
assembly_id | int | Assembly identification number. Is unique within a database.
clr_name | nvarchar(4000) | Canonical string that encodes the simple name, version number, culture, public key, and architecture of the assembly. This value uniquely identifies the assembly on the common language runtime (CLR) side.
permission_set | tinyint | Permission-set/security-level for assembly. 1 = Safe Access 2 = External Access 3 = Unsafe Access
permission_set_desc | nvarchar(60) | Description for permission-set/security-level for assembly. SAFE_ACCESS EXTERNAL_ACCESS UNSAFE_ACCESS
is_visible | bit | 1 = Assembly is visible to register Transact-SQL entry points. 0 = Assembly is intended only for managed callers. That is, the assembly provides internal implementation for other assemblies in the database.
create_date | datetime | Date the assembly was created or registered.
modify_date | datetime | Date the assembly was modified.
is_user_defined | bit | Indicates the source of the assembly. 0 = System-defined assemblies (such as Microsoft.SqlServer.Types for the hierarchyid data type) 1 = User-defined assemblies

Permissions

The visibility of the metadata in catalog views is limited to securables that a user either owns or on which the user has been granted some permission.

See Also

Reference

sys.all_views (SQL Server PDW)
sys.assembly_types (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.assembly_modules (SQL Server PDW)

[image]

sys.assembly_modules (SQL Server PDW)
See Also Send Feedback

Returns one row for each function, or procedure that is defined by a common language runtime (CLR) assembly. This catalog view maps CLR stored procedures, or CLR functions to their underlying implementation. Objects of type TA, AF, PC, FS, and FT have an associated assembly module. To find the association between the object and the assembly, you can join this catalog view to other catalog views. For example, when you create a CLR stored procedure, it is represented by one row in sys.objects, one row in sys.procedures (which inherits from sys.objects), and one row in sys.assembly_modules. The stored procedure itself is represented by the metadata in sys.objects and sys.procedures. References to the procedure’s underlying CLR implementation are found in sys.assembly_modules.

Column name

Data type

Description

object_id

int

Object identification number of the SQL object. Is unique within a database.

assembly_id

int

ID of the assembly from which this module was created.

assembly_class

sysname

Name of the class within the assembly that defines this module.

assembly_method

sysname

Name of the method within the assembly_class that defines this module.

NULL for aggregate functions (AF).

null_on_null_input

bit

Module was declared to produce a NULL output for any NULL input.

execute_as_principal_id

int

ID of the database principal under which the context execution occurs, as specified by the EXECUTE AS clause of the CLR function, OR stored procedure.

NULL = EXECUTE AS CALLER. This is the default.

ID of the specified database principal = EXECUTE AS SELF, EXECUTE AS user_name, or EXECUTE AS login_name.

-2 = EXECUTE AS OWNER.

Permissions

The visibility of the metadata in catalog views is limited to securables that a user either owns or on which the user has been granted some permission. For more information, see Metadata Visibility Configuration.

See Also

Reference

System Views (SQL Server PDW)
sys.all_sql_modules (SQL Server PDW)


sys.assembly_types (SQL Server PDW)

[image]

sys.assembly_types (SQL Server PDW)
See Also Send Feedback

Contains a row for each user-defined type that is defined by a CLR assembly. The following sys.assembly_types appear in the list of inherited columns (see sys.types (SQL Server PDW)) after rule_object_id.

Column name | Data type | Description
assembly_id | int | ID of the assembly from which this type was created.
assembly_class | sysname | Name of the class within the assembly that defines this type.
is_binary_ordered | bit | Sorting the bytes of this type is equivalent to sorting using comparison operators on the type.
is_fixed_length | bit | Length of the type is always the same as max_length.
prog_id | nvarchar(40) | ProgID of the type as exposed to COM.
assembly_qualified_name | nvarchar(4000) | Assembly qualified type name. The name is in a format suitable to be passed to Type.GetType().

Permissions

The visibility of the metadata in catalog views is limited to securables that a user either owns or on which the user has been granted some permission.

See Also

Reference

sys.all_views (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.check_constraints (SQL Server PDW)

[image]

sys.check_constraints (SQL Server PDW)
See Also Send Feedback

This sys.check_constraints Catalog view is available in SQL Server PDW. However, this view will return 0 rows.

Column Name | Data Type | Description
<Columns inherited from sys.objects> |  | Return 0 rows.
is_disabled | bit |
is_not_for_replication | bit |
is_not_trusted | bit |
parent_column_id | int |
definition | nvarchar(max) |
uses_database_collation | bit |
is_system_named | bit |

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.certificates (SQL Server PDW)

[image]

sys.certificates (SQL Server PDW)
See Also Send Feedback

Returns a row for each certificate in the database.

Column name | Data type | Description
name | sysname | Name of the certificate. Is unique within the database.
certificate_id | int | ID of the certificate. Is unique within the database.
principal_id | int | ID of the database principal that owns this certificate.
pvt_key_encryption_type | char(2) | How the private key is encrypted. NA = There is no private key for the certificate MK = Private key is encrypted by the master key PW = Private key is encrypted by a user-defined password SK = Private key is encrypted by the service master key.
pvt_key_encryption_type_desc | nvarchar(60) | Description of how the private key is encrypted. NO_PRIVATE_KEY ENCRYPTED_BY_MASTER_KEY ENCRYPTED_BY_PASSWORD ENCRYPTED_BY_SERVICE_MASTER_KEY
is_active_for_begin_dialog | bit | If 1, this certificate is used to initiate encrypted service dialogs.
issuer_name | nvarchar(442) | Name of certificate issuer.
cert_serial_number | nvarchar(64) | Serial number of certificate.
sid | varbinary(85) | Login SID for this certificate.
string_sid | nvarchar(128) | String representation of the login SID for this certificate
subject | nvarchar(4000) | Subject of this certificate.
expiry_date | datetime | When certificate expires.
start_date | datetime | When certificate becomes valid.
thumbprint | varbinary(32) | SHA-1 hash of the certificate. The SHA-1 hash is globally unique.
attested_by | nvarchar(260) | System use only.
pvt_key_last_backup_date | datetime | The date and time the certificate’s private key was last exported.

Permissions

The visibility of the metadata in catalog views is limited to securables that a user either owns or on which the user has been granted some permission.

See Also

Reference

CREATE CERTIFICATE (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.columns (SQL Server PDW)

[image]

sys.columns (SQL Server PDW)
See Also Send Feedback

Shows columns for user-defined tables and user-defined views.

Column Name | Data Type | Description | Range
object_id | int | ID of the object to which this column belongs. |
name | sysname | Name of the column. Unique in object. |
column_id | int | ID of the column. Unique in object. |
system_type_id | tinyint | ID of the system type of the column. |
user_type_id | int | ID of the type of the column as defined by the user. |
max_length | smallint | Maximum length (in bytes) of the column. | Includes -1 (not valid) for unsupported column types.
precision | tinyint | Precision of the column if numeric-based; otherwise, 0. |
scale | tinyint | Scale of column if numeric-based; otherwise, 0. |
collation_name | sysname | Name of the collation of the column if character-based; otherwise, NULL. |
is_nullable | bit | 1 = Column is nullable. |
is_ansi_padded | bit | 1 = Column uses ANSI_PADDING ON behavior if character, binary, or variant. | Always 0.
is_rowguidcol | bit | 1 = Column is a declared ROWGUIDCOL. | Always 0.
is_identity | bit | 1 = Column has identity values. | Always 0.
is_computed | bit | 1 = Column is a computed column. | Always 0.
is_filestream | bit | 1 = Column is a FILESTREAM column. | Always 0.
is_replicated | bit | 1 = Column is replicated. | Always 0.
is_non_sql_subscribed | bit | 1 = Column has a non-SQL subscriber. | Always 0.
is_merge_published | bit | 1 = Column is merge-published. | Always 0.
is_dts_replicated | bit | 1 = Column is replicated by using SSIS. | Always 0.
is_xml_document | bit | 1 = Content is a complete XML document. | Always 0.
xml_collection_id | int | 0 = No XML schema collection. | Always 0.
default_object_id | int | ID of the default object; 0 = No default. | Always 0.
rule_object_id | int | ID of the stand-alone rule bound to the column. 0 = No stand-alone rule. | Always 0.
is_sparse | bit | 1 = Column is a sparse column. | Always 0.
is_column_set | bit | 1 = Column is a column set. | Always 0.

See Also

Reference

sys.all_columns (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.computed_columns (SQL Server PDW)

[image]

sys.computed_columns (SQL Server PDW)
See Also Send Feedback

Contains a row for each column found in sys.columns that is a computed column. Since SQL Server PDW does not support computed column, this view returns 0 rows.

For more information about this view, see the sys.computed_columns (Transact-SQL) documentation on MSDN.

See Also


sys.data_spaces (SQL Server PDW)

[image]

sys.data_spaces (SQL Server PDW)
See Also Send Feedback

Contains a row for each data space. This can be a filegroup or partition scheme.

Column Name | Data Type | Description | Range
name | sysname | Name of data space, unique within the database. |
data_space_id | int | Data space ID number, unique within the database. |
type | char(2) | Data space type: FG = Filegroup PS = Partition scheme |
type_desc | nvarchar(60) | Description of data space type: ROWS_FILEGROUP PARTITION_SCHEME |
is_default | bit | 1 = This is the default data space. 0 = This is not the default data space. |
is_system | bit | 1 = Data space is used for full-text index fragments. 0 = Data space is not used for full-text index fragments. |

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.database_files (SQL Server PDW)

[image]

sys.database_files (SQL Server PDW)
See Also Send Feedback

Returns one row per file of the current SQL Server PDW database as stored in the database itself. This is a per-database view.

Column Name | Data Type | Description | Range
file_id | int | ID of the file within database. |
file_guid | uniqueidentifier | GUID for the file. |
type | tinyint | File type. | 0 = Rows (distributed or replicated) 1 = Log
type_desc | nvarchar(60) | Description of the file type. | 'ROWS','LOG'
data_space_id | int | Value can be 0 or greater than 0. A value of 0 represents the database log file, and a value greater than 0 represents the ID of the filegroup where this data file is stored. |
name | sysname | Logical name of the file in the database. |
physical_name | nvarchar(260) | Operating system file name. |
state | tinyint | File state: 0 = ONLINE 1 = RESTORING 2 = RECOVERING 3 = RECOVERY_PENDING 4 = SUSPECT 5 = Not supported. 6 = OFFLINE 7 = DEFUNCT |
state_desc | nvarchar(60) | Description of the file state: ONLINE RESTORING RECOVERING RECOVERY_PENDING SUSPECT OFFLINE DEFUNCT |
size | int | Always 0. The size column is not supported in this release. |
max_size | int | Maximum file size, in 8 KB pages. 0 = No growth is allowed. -1 = File will grow until the disk is full. 268435456 = Log file will grow to a maximum size of 2 TB. Note Databases that are upgraded with an unlimited log file size will report -1 for the maximum size of the log file. |
growth | int | 0 = File is fixed size and will not grow. >0 = File will grow automatically. If is_percent_growth = 0, growth increment is in units of 8 KB pages, rounded to the nearest 64 KB. If is_percent_growth = 1, growth increment is expressed as a whole number percentage. |
is_media_read_only | bit | 1 = File is on read-only media. 0 = File is on read-write media. | Always 0.
is_read_only | bit | 1 = File is marked read-only. 0 = File is marked read/write. | Always 0.
is_sparse | bit | 1 = File is a sparse file. 0 = File is not a sparse file. | Always 0.
is_percent_growth | bit | 1 = Growth of the file is a percentage. 0 = Absolute growth size in pages. |
is_name_reserved | bit | 1 = Dropped file name (name or physical_name) is reusable only after the next log backup. When files are dropped from a database, the logical names stay in a reserved state until the next log backup. This column is relevant only under the full recovery model and the bulk-logged recovery model. |
create_lsn | numeric(25,0) | Log sequence number (LSN) at which the file was created. |
drop_lsn | numeric (25,0) | LSN at which the file was dropped. 0 = File name is unavailable for reuse. |
read_only_lsn | numeric (25,0) | LSN at which the filegroup that contains the file changed from read/write to read-only (most recent change). |
read_write_lsn | numeric (25,0) | LSN at which the filegroup that contains the file changed from read-only to read/write (most recent change). |
differential_base_lsn | numeric (25,0) | Base for differential backups. Data extents changed after this LSN will be included in a differential backup. |
differential_base_guid | uniqueidentifier | Unique identifier of the base backup on which a differential backup will be based. |
differential_base_time | datetime | Time corresponding to differential_base_lsn. |
redo_start_lsn | numeric(25,0) | LSN at which the next roll forward must start. Is NULL unless state = RESTORING or state = RECOVERY_PENDING. |
redo_start_fork_guid | uniqueidentifier | Unique identifier of the recovery fork. The first_fork_guid of the next log backup restored must match this value. This represents the current state of the file. |
redo_target_lsn | numeric(25,0) | LSN at which the online roll forward on this file can stop. Is NULL unless state = RESTORING or state = RECOVERY_PENDING. |
redo_target_fork_guid | uniqueidentifier | The recovery fork on which the file can be recovered. Paired with redo_target_lsn. |
backup_lsn | numeric(25,0) | The LSN of the most recent data or differential backup of the file. |

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.database_permissions (SQL Server PDW)

[image]

sys.database_permissions (SQL Server PDW)
See Also Send Feedback

Returns a row for every permission or column-exception permission in the database. For columns, there is a row for every permission that is different from the corresponding object-level permission. If the column permission is the same as the corresponding object permission, there will be no row for it and the actual permission used will be that of the object.

Important note Important

Column-level permissions override object-level permissions on the same entity.

Column name | Data type | Description
class | tinyint | Identifies class on which permission exists. 0 = Database 1 = Object or Column 3 = Schema 4 = Database Principal 5 = Assembly (Not available in PDW) 6 = Type 10 = XML Schema Collection (Not available in PDW) (Not available in PDW) 15 = Message Type (Not available in PDW) 16 = Service Contract (Not available in PDW) 17 = Service (Not available in PDW) 18 = Remote Service Binding (Not available in PDW) 19 = Route (Not available in PDW) 23 = Full-Text Catalog (Not available in PDW) 24 = Symmetric Key (Not available in PDW) 25 = Certificate (Not available in PDW) 26 = Asymmetric Key (Not available in PDW)
class_desc | nvarchar(60) | Description of class on which permission exists. DATABASE OBJECT_OR_COLUMN SCHEMA DATABASE_PRINCIPAL ASSEMBLY TYPE XML_SCHEMA_COLLECTION MESSAGE_TYPE SERVICE_CONTRACT SERVICE REMOTE_SERVICE_BINDING ROUTE FULLTEXT_CATALOG SYMMETRIC_KEY CERTIFICATE ASYMMETRIC_KEY
major_id | int | ID of thing on which permission exists, interpreted according to class. For most, this is simply the kind of ID that applies to what the class represents. Interpretation for nonstandard is as follows: 0 = Always 0 1 = Object-ID Negative IDs are assigned to system objects.
minor_id | int | Secondary-ID of thing on which permission exists, interpreted according to class. For most, this is zero. Otherwise, it is the following: 1 = Column-ID if a column. Otherwise, it is 0 if an object.
grantee_principal_id | int | Database principal ID to which the permissions are granted.
grantor_principal_id | int | Database principal ID of the grantor of these permissions.
type | char(4) | Database permission type. For a list of permission types, see the next table.
permission_name | nvarchar(128) | Permission name.
state | char(1) | Permission state: D = Deny R = Revoke G = Grant W = Grant With Grant Option
state_desc | nvarchar(60) | Description of permission state: DENY REVOKE GRANT GRANT_WITH_GRANT_OPTION
Permission type | Permission name | Applies to securable
AL | ALTER | DATABASE, OBJECT, ROLE, SCHEMA, USER
ALDS | ALTER ANY DATASPACE | DATABASE
ALED | ALTER ANY DATABASE EVENT NOTIFICATION | DATABASE
ALRL | ALTER ANY ROLE | DATABASE
ALUS | ALTER ANY USER | DATABASE
BADB | BACKUP DATABASE | DATABASE
CL | CONTROL | DATABASE, OBJECT, ROLE, SCHEMA, USER
CO | CONNECT | DATABASE
CRPR | CREATE PROCEDURE | DATABASE
CRRL | CREATE ROLE | DATABASE
CRTB | CREATE TABLE | DATABASE
CRVW | CREATE VIEW | DATABASE
DL | DELETE | DATABASE, OBJECT, SCHEMA
EX | EXECUTE | DATABASE, OBJECT, SCHEMA
IM | IMPERSONATE | USER
IN | INSERT | DATABASE, OBJECT, SCHEMA
RC | RECEIVE | OBJECT
SL | SELECT | DATABASE, OBJECT, SCHEMA
SPLN | SHOWPLAN | DATABASE
TO | TAKE OWNERSHIP | DATABASE, OBJECT, ROLE, SCHEMA
UP | UPDATE | DATABASE, OBJECT, SCHEMA
VW | VIEW DEFINITION | DATABASE, OBJECT, ROLE, SCHEMA, USER

Permissions

Any user can see his/her own permissions. To see permissions for other users requires VIEW DEFINITION, ALTER ANY USER, or any permission on a user. To see user-defined roles requires ALTER ANY ROLE, or membership in the role (such as public).

Examples

A: Listing all the permissions of database principals

The following query lists the permissions explicitly granted or denied to database principals.

Important note Important

The permissions of fixed database roles do not appear in sys.database_permissions. Therefore, database principals may have additional permissions not listed here.

Copy image Copy Code
SELECT pr.principal_id, pr.name, pr.type_desc,
    pr.authentication_type_desc, pe.state_desc, pe.permission_name
FROM sys.database_principals AS pr
JOIN sys.database_permissions AS pe
    ON pe.grantee_principal_id = pr.principal_id;

B: Listing permissions on schema objects within a database

The following query joins sys.database_principals and sys.database_permissions to sys.objects and sys.schemas to list permissions granted or denied to specific schema objects.

Copy image Copy Code
SELECT pr.principal_id, pr.name, pr.type_desc,
    pr.authentication_type_desc, pe.state_desc,
    pe.permission_name, s.name + '.' + o.name AS ObjectName
FROM sys.database_principals AS pr
JOIN sys.database_permissions AS pe
    ON pe.grantee_principal_id = pr.principal_id
JOIN sys.objects AS o
    ON pe.major_id = o.object_id
JOIN sys.schemas AS s
    ON o.schema_id = s.schema_id;

See Also

Reference

sys.database_principals (SQL Server PDW)
sys.server_permissions (SQL Server PDW)
sys.server_principals (SQL Server PDW)


sys.database_principals (SQL Server PDW)

[image]

sys.database_principals (SQL Server PDW)
See Also Send Feedback

Returns a row for each security principal in a database.

Column name

Data type

Description

name

sysname

Name of principal, unique within the database.

principal_id

int

ID of principal, unique within the database.

type

char(1)

Principal type:

S = SQL user

U = Windows user

G = Windows group

A = Application role (Not available in PDW)

R = Database role

C = User mapped to a certificate (Not available in PDW)

K = User mapped to an asymmetric key (Not available in PDW)

type_desc

nvarchar(60)

Description of principal type.

SQL_USER

WINDOWS_USER

WINDOWS_GROUP

APPLICATION_ROLE (Not available in PDW)

DATABASE_ROLE

CERTIFICATE_MAPPED_USER (Not available in PDW)

ASYMMETRIC_KEY_MAPPED_USER (Not available in PDW)

default_schema_name

sysname

Name to be used when SQL name does not specify schema. Null for principals not of type S, U, or A.

create_date

datetime

Time at which the principal was created.

modify_date

datetime

Time at which the principal was last modified.

owning_principal_id

int

ID of the principal that owns this principal. All principals except Database Roles must be owned by dbo.

sid

varbinary(85)

SID (Security Identifier) of the principal. NULL for SYS and INFORMATION SCHEMAS

is_fixed_role

bit

If 1, then this row represents an entry for one of the fixed database roles: db_owner, db_accessadmin, db_datareader, db_datawriter, db_ddladmin, db_securityadmin, db_backupoperator, db_denydatareader, db_denydatawriter.

authentication_type

int

Signifies authentication type.

Value | Description
0 | No authentication
1 | Instance authentication
2 | Database authentication
3 | Windows Authentication

authentication_type_desc_

nvarchar(60)

Description of the authentication type.

Value | Description
NONE | No authentication
INSTANCE | Instance authentication
DATABASE | Database authentication
WINDOWS | Windows Authentication

Default_language_name

sysname

Signifies the default language for this principal. Always NULL in SQL Server PDW.

Default_language_lcid

int

Signifies the default LCID for this principal. Always NULL in SQL Server PDW.

Permissions

Any user can see their own user name, the system users, and the fixed database roles. To see other users, requires ALTER ANY USER, or a permission on the user. To see user-defined roles, requires ALTER ANY ROLE, or membership in the role.

Examples

A: Listing all the permissions of database principals

The following query lists the permissions explicitly granted or denied to database principals.

Important note Important

The permissions of fixed database roles do not appear in sys.database_permissions. Therefore, database principals may have additional permissions not listed here.

Copy image Copy Code
SELECT pr.principal_id, pr.name, pr.type_desc,
    pr.authentication_type_desc, pe.state_desc, pe.permission_name
FROM sys.database_principals AS pr
JOIN sys.database_permissions AS pe
    ON pe.grantee_principal_id = pr.principal_id;

B: Listing permissions on schema objects within a database

The following query joins sys.database_principals and sys.database_permissions to sys.objects and sys.schemas to list permissions granted or denied to specific schema objects.

Copy image Copy Code
SELECT pr.principal_id, pr.name, pr.type_desc,
    pr.authentication_type_desc, pe.state_desc,
    pe.permission_name, s.name + '.' + o.name AS ObjectName
FROM sys.database_principals AS pr
JOIN sys.database_permissions AS pe
    ON pe.grantee_principal_id = pr.principal_id
JOIN sys.objects AS o
    ON pe.major_id = o.object_id
JOIN sys.schemas AS s
    ON o.schema_id = s.schema_id;

See Also

Reference

System Views (SQL Server PDW)
sys.server_principals (SQL Server PDW)
sys.sql_logins (SQL Server PDW)


sys.database_role_members (SQL Server PDW)

[image]

sys.database_role_members (SQL Server PDW)
See Also Send Feedback

Returns one row for each member of each database role for SQL Server PDW.

Column name | Data type | Description
role_principal_id | int | Database Principal ID of the role.
member_principal_id | int | Database Principal ID of the member.

Permissions

Any user can view their own role membership. To view other role memberships requires membership in the db_securityadmin fixed database role or VIEW DEFINITION on the database.

Examples

The following query returns the members of the database roles.

Copy image Copy Code
SELECT DP1.name AS DatabaseRoleName,
   isnull (DP2.name, 'No members') AS DatabaseUserName
 FROM sys.database_role_members AS DRM
 RIGHT OUTER JOIN sys.database_principals AS DP1
   ON DRM.role_principal_id = DP1.principal_id
 LEFT OUTER JOIN sys.database_principals AS DP2
   ON DRM.member_principal_id = DP2.principal_id
ORDER BY DP1.name;

See Also

Reference

System Views (SQL Server PDW)
sys.database_principals (SQL Server PDW)


sys.databases (SQL Server PDW)

[image]

sys.databases (SQL Server PDW)
See Also Send Feedback

The sys.databases view is aligned with the corresponding view exposed by SQL Server. SQL Server PDW exposes logical databases rather than the actual physical databases on the various Compute node instances. Because some features are not supported, some columns have fixed return values.

Column Name

Data Type

Description

Range

name

sysname

Name of database, unique within an instance of SQL Server.

database_id

int

ID of the database, unique within an instance of SQL Server.

source_database_id

int

NULL = Not a database snapshot.

Always NULL.

owner_sid

varbinary(85)

SID (SecurityIdentifier) of the external owner of the database, as registered to the server.

create_date

datetime

Date the database was created or renamed. For tempdb, this value changes every time the server restarts.

compatibility_level

tinyint

Integer corresponding to the version of SQL Server for which behavior is compatible.

Always returns 100.

collation_name

sysname

Collation for the database. Acts as the default collation in the database.

Always Latin1_General_100_CI_AS_KS_WS.

user_access

tinyint

User access setting:

0 = MULTI_USER specified

Always 0.

user_access_desc

nvarchar(60)

Description of user-access setting: MULTI_USER

Always ‘MULTI_USER’.

is_read_only

bit

0 = Database is READ_WRITE.

Always 0.

is_auto_close_on

bit

0 = AUTO_CLOSE is OFF.

Always 0.

is_auto_shrink_on

bit

0 = AUTO_SHRINK is OFF.

Always 0.

state

tinyint

Database state:

0 = ONLINE

Always 0.

state_desc

nvarchar(60)

Description of the database state: ONLINE

‘ONLINE’

is_in_standby

bit

Database is read-only for restore log.

Always 0.

is_cleanly_shutdown

bit

0 = Database did not shut down cleanly; recovery is required on startup.

Always 0.

is_supplemental_logging_enabled

bit

0 = SUPPLEMENTAL_LOGGING is OFF.

Always 0.

snapshot_isolation_state

tinyint

State of snapshot-isolation transactions being allowed,0 = Snapshot isolation state is OFF (default). Snapshot isolation is disallowed.

Always 0.

snapshot_isolation_state_desc

nvarchar(60)

Description of state of snapshot-isolation transactions being allowed

‘OFF’

is_read_committed_snapshot_on

bit

0 = option is OFF.

Always 0.

recovery_model

tinyint

Recovery model selected:

1 = FULL

3 = SIMPLE

recovery_model_desc

nvarchar(60)

Description of recovery model selected.

‘FULL’ or ‘SIMPLE’

page_verify_option

tinyint

Setting of PAGE_VERIFY option:

2 = CHECKSUM

Always 2.

page_verify_option_desc

nvarchar(60)

Description of PAGE_VERIFY option setting

‘CHECKSUM’

is_auto_create_stats_on

bit

1 = AUTO_CREATE_STATISTICS is ON.

This is the setting on the Compute nodes, and does not indicate the setting for the Control node.

Always 1.

is_auto_update_stats_on

bit

1 = AUTO_UPDATE_STATISTICS is ON.

This is the setting on the Compute nodes, and does not indicate the setting for the Control node.

Always 1.

is_auto_update_stats_async_on

bit

0 = AUTO_UPDATE_STATISTICS_ASYNC is OFF.

This is the setting on the Compute nodes, and does not indicate the setting for the Control node.

Always 0.

is_ansi_null_default_on

bit

0 = ANSI_NULL_DEFAULT is OFF.

Always 0.

is_ansi_nulls_on

bit

0 = ANSI_NULLS is OFF.

Always 0.

is_ansi_padding_on

bit

0 = ANSI_PADDING is OFF.

Always 0.

is_ansi_warnings_on

bit

0 = ANSI_WARNINGS is OFF.

Always 0.

is_arithabort_on

bit

0 = ARITHABORT is OFF.

Always 0.

is_concat_null_yields_null_on

bit

0 = CONCAT_NULL_YIELDS_NULL is OFF.

Always 0.

is_numeric_roundabort_on

bit

0 = NUMERIC_ROUNDABORT is OFF.

Always 0.

is_quoted_identifier_on

bit

0 = QUOTED_IDENTIFIER is OFF.

Always 0.

is_recursive_triggers_on

bit

0 = RECURSIVE_TRIGGERS is OFF.

Always 0.

is_cursor_close_on_commit_on

bit

0 = CURSOR_CLOSE_ON_COMMIT is OFF.

Always 0.

is_local_cursor_default

bit

0 = CURSOR_DEFAULT is global.

Always 0.

is_fulltext_enabled

bit

0 = Full-text is disabled for the database.

Always 0.

is_trustworthy_on

bit

0 = Database has not been marked trustworthy.

Always 0.

is_db_chaining_on

bit

0 = Cross-database ownership chaining is OFF.

Always 0.

is_parameterization_forced

bit

0 = Parameterization is SIMPLE.

Always 0.

is_master_key_encrypted_by_server

bit

Indicates if the database has an encrypted master key. An encrypted master key is one requirement for TDE.

0 = The database does not have an encrypted master key.

1 = The database has an encrypted master key.

is_published

bit

0 = Is not a publication database.

Always 0.

is_subscribed

bit

0 = Is not a subscription database.

Always 0.

is_merge_published

bit

0 = Is not a publication database in a merge replication topology.

Always 0.

is_distributor

bit

0 = Is not the distribution database for a replication topology.

Always 0.

is_sync_with_backup

bit

0 = Is not marked for replication synchronization with backup.

Always 0.

service_broker_guid

uniqueidentifier

Identifier of the service broker for this database. Used as the broker_instance of the target in the routing table.

NULL

is_broker_enabled

bit

0 = Broker disabled.

Always 0.

log_reuse_wait

tinyint

Reuse of transaction log space is currently waiting on one of the following: 0 = Nothing

Always 0.

log_reuse_wait_desc

nvarchar(60)

Description of reuse of transaction log space.

Always return ‘NOTHING’.

is_date_correlation_on

bit

0 = OFF

Always 0.

is_cdc_enabled

bit

1 = Database is enabled for change data capture.

Always 0.

is_encrypted

bit

Indicates whether the database is encrypted by using TDE.

0 = Not Encrypted

1 = Encrypted

is_honor_broker_priority_on

bit

Indicates whether the database honors conversation priorities

0 = OFF

Always 0.

replica_id

uniqueidentifier

Unique identifier of the local AlwaysOn Availability Groups availability replica of the availability group, if any, in which the database is participating.

NULL = database is not part of an availability replica of in availability group.

Always NULL.

group_database_id

uniqueidentifier

Unique identifier of the database within an AlwaysOn availability group, if any, in which the database is participating. group_database_id is the same for this database on the primary replica and on every secondary replica on which the database has been joined to the availability group.

NULL = database is not part of an availability replica in any availability group.

Always NULL.

default_language_lcid

smallint

Indicates the local id (lcid) of the default language of a contained database.

Note Functions as the Configure the default language Server Configuration Option of sp_configure. This value is null for a non-contained database.

Always NULL.

default_language_name

nvarchar(128)

Indicates the default language of a contained database.

This value is null for a non-contained database.

Always NULL.

default_fulltext_language_lcid

int

Indicates the local id (lcid) of the default fulltext language of the contained database.

Note Functions as the default Configure the default full-text language Server Configuration Option of sp_configure. This value is null for a non-contained database.

Always NULL.

default_fulltext_language_name

nvarchar(128)

Indicates the default fulltext language of the contained database.

This value is null for a non-contained database.

Always NULL.

is_nested_triggers_on

bit

Indicates whether or not nested triggers are allowed in the contained database.

0 = nested triggers are not allowed

1 = nested triggers are allowed

Note Functions as the Configure the nested triggers Server Configuration Option of sp_configure. This value is null for a non-contained database. See sys.configurations for further information.

Always NULL.

is_transform_noise_words_on

bit

Indicates whether or noise words should be transformed in the contained database.

0 = noise words should not be transformed.

1 = noise words should be transformed.

Note Functions as the transform noise words of sp_configure. This value is null for a non-contained database. See sys.configurations for further information.

Always NULL.

two_digit_year_cutoff

smallint

Indicates a value of a number between 1753 and 9999 to represent the cutoff year for interpreting two-digit years as four-digit years.

Note Functions as the Configure the two digit year cutoff Server Configuration Option of sp_configure. This value is null for a non-contained database. See sys.configurations for further information.

Always NULL.

containment

tinyint not null

Indicates the containment status of the database.

Always 0.

containment_desc

nvarchar(60) not null

Indicates the containment status of the database.

NONE = legacy database (zero containment)

PARTIAL = partially contained database

Always NONE.

target_recovery_time_in_seconds

int

The estimated time to recover the database, in seconds. Nullable.

Always 0.

See Also

Reference

System Views (SQL Server PDW)


sys.default_constraints (SQL Server PDW)

[image]

sys.default_constraints (SQL Server PDW)
See Also Send Feedback

Contains a row for each column that has a constraint (default value) created as part of a CREATE TABLE or ALTER TABLE statement in SQL Server PDW.

Column Name | Data Type | Description
<Columns inherited from sys.objects> |  | For a list of the inherited columns in the view, see sys.objects (SQL Server PDW).
parent_column_id | int | ID of the column in parent_object_id to which this constraint applies.
Definition | nvarchar(4000) | SQL expression that defines this constraint.
is_system_named | bit | 0 = User specified the constraint name. 1 = System generated the constraint name.

See Also

Reference

System Views (SQL Server PDW)


sys.extended_properties (SQL Server PDW)

[image]

sys.extended_properties (SQL Server PDW)
See Also Send Feedback

Returns a row for each extended property in the current SQL Server PDW database. In this release, the result is 0 rows.

For more information about this view, see the sys.extended_properties (Transact-SQL) documentation on MSDN.

See Also


sys.filegroups (SQL Server PDW)

[image]

sys.filegroups (SQL Server PDW)
See Also Send Feedback

Contains a row for each data space that is a filegroup.

Column Name | Data Type | Description | Range
<inherited columns from data spaces> | -- | Information not available. |
filegroup_guid | uniqueidentifier | Information not available. |
log_filegroup_id | int | Identified for informational purposes only. Not supported. | Returns NULL.
is_read_only | bit | 1 = Filegroup is read-only. 0 = Filegroup is read/write | Returns 0.

See Also

Reference

System Views (SQL Server PDW)


sys.foreign_key_columns (SQL Server PDW)

[image]

sys.foreign_key_columns (SQL Server PDW)
See Also Send Feedback

SQL Server PDW does not support foreign key columns, but it supports this view to allow integration with tools and applications. This view returns 0 rows.

Column name | Data type | Description
constraint_object_id | int | Return 0 rows.
constraint_column_id | int |
parent_object_id | int |
parent_column_id | int |
referenced_object_id | int |
referenced_column_id | int |

See Also

Reference

System Views (SQL Server PDW)


sys.foreign_keys (SQL Server PDW)

[image]

sys.foreign_keys (SQL Server PDW)
See Also Send Feedback

SQL Server PDW does not support foreign keys, but it supports this view to allow integration of tools and applications. This view returns 0 rows.

Column Name | Data Type | Description
<Columns inherited from sys.objects> |  | Returns 0 rows.
referenced_object_id | int |
key_index_id | int |
is_disabled | bit |
is_not_for_replication | bit |
is_not_trusted | bit |
delete_referential_action | tinyint |
delete_referential_action_desc | nvarchar(60) |
update_referential_action | tinyint |
update_referential_action_desc | nvarchar(60) |
is_system_named | bit |

See Also

Reference

System Views (SQL Server PDW)


sys.identity_columns (SQL Server PDW)

[image]

sys.identity_columns (SQL Server PDW)
See Also Send Feedback

SQL Server PDW does not support identity columns, but it supports this view to allow integration with tools and applications. This view returns 0 rows.

Column Name | Data Type | Description
<columns inherited from sys.columns> |  | Returns 0 rows.
seed_value | sql_variant |
increment_value | sql_variant |
last_value | sql_variant |
is_not_for_replication | bit |
is_computed | bit |
is_sparse | bit |
is_column_set | bit |

See Also

Reference

System Views (SQL Server PDW)


sys.index_columns (SQL Server PDW)

[image]

sys.index_columns (SQL Server PDW)
See Also Send Feedback

Contains one row per column that is part of a sys.indexes index or unordered table.

Column Name | Data Type | Description | Range
object_id | int | id of the object on which the index is defined. |
index_id | int | id of the index in which the column is defined. |
index_column_id | int | id of the index column. index_column_id is unique only within index_id. |
column_id | int | id of the column in object_id. column_id is unique only within object_id. 0 = Row Identifier (RID) in a nonclustered index. |
key_ordinal | tinyint | Ordinal (1-based) within set of key columns. 0 = Not a key column. | Always > 0 for columns in rowstore indexes because all indexed columns are key columns. Always 0 for columns in clustered columnstore index columns because all columns are included columns.
partition_ordinal | tinyint | 1 = partition column. 0 = Not the partition column. |
is_descending_key | bit | 1 = rowstore index column is sorted in descending order. 0 = rowstore index column is sorted in ascending order. 0 = column is in a clustered columnstore index. |
is_included_column | bit | 0 = Column is not an included column. 1 = Column is an included column. | Always 0 for rowstore index columns because all indexed columns are key columns. Always 1 for clustered columnstore index columns because all columns are included columns.

See Also

Reference

System Views (SQL Server PDW)


sys.indexes (SQL Server PDW)

[image]

sys.indexes (SQL Server PDW)
See Also Send Feedback

Returns indexes for SQL Server PDW.

Column Name | Data Type | Description | Range
object_id | int | id of the object to which this index belongs. |
name | sysname | Name of the index. Name is unique only within the object. NULL = Heap |
index_id | int | id of the index. index_id is unique only within the object. 0 = Heap 1 = Clustered index > 1 = Non-clustered index |
type | tinyint | Type of index: 0 = Heap 1 = Clustered 2 = Non-clustered 5 = Clustered xVelocity memory optimized columnstore index
type_desc | nvarchar(60) | Description of index type: ‘HEAP’ ‘CLUSTERED’ ‘NONCLUSTERED’ ‘CLUSTERED COLUMNSTORE’ |
is_unique | bit | 0 = Index is not unique. | Always 0.
data_space_id | int | id of the data space for this index. Data space is either a filegroup or partition scheme. 0 = object_id is a table-valued function. |
ignore_dup_key | bit | 0 = IGNORE_DUP_KEY is OFF. | Always 0.
is_primary_key | bit | 1 = Index is part of a PRIMARY KEY constraint. | Always 0.
is_unique_constraint | bit | 1 = Index is part of a UNIQUE constraint. | Always 0.
fill_factor | tinyint | > 0 = FILLFACTOR percentage used when the index was created or rebuilt. 0 = Default value | Always 0.
is_padded | bit | 0 = PADINDEX is OFF. | Always 0.
is_disabled | bit | 1 = Index is disabled. 0 = Index is not disabled. |
is_hypothetical | bit | 0 = Index is not hypothetical. | Always 0.
allow_row_locks | bit | 1 = Index allows row locks. | Always 1.
allow_page_locks | bit | 1 = Index allows page locks. | Always 1.
has_filter | bit | 0 = Index does not have a filter. | Always 0.
filter_definition | nvarchar(max) | Expression for the subset of rows included in the filtered index. | Always NULL.

See Also

Reference

System Views (SQL Server PDW)


sys.key_constraints (SQL Server PDW)

[image]

sys.key_constraints (SQL Server PDW)
See Also Send Feedback

SQL Server PDW does not support primary keys or unique constraints, but it does support this view for integration of tools and applications. This view will return 0 rows.

Column Name | Data Type | Description
<Columns inherited from sys.objects> |  | Return 0 rows.
unique_index_id | int |
is_system_named | bit |

See Also

Reference

System Views (SQL Server PDW)


sys.master_files (SQL Server PDW)

[image]

sys.master_files (SQL Server PDW)
See Also Send Feedback

Contains a row per file of a database as stored in the master database. This is a single, system-wide view.

Column name

Data type

Description

database_id

int

ID of the database to which this file applies. The master database_id is always 1.

file_id

int

ID of the file within database. The primary file_id is always 1.

file_guid

uniqueidentifier

Unique identifier of the file.

NULL = Database was upgraded from an earlier version of SQL Server.

type

tinyint

File type:

0 = Rows. (Includes files of full-text catalogs that are upgraded to or created in SQL Server 2014.)

1 = Log

2 = FILESTREAM

3 = Identified for informational purposes only. Not supported. Future compatibility is not guaranteed.

4 = Full-text (Full-text catalogs earlier than SQL Server 2014; full-text catalogs that are upgraded to or created in SQL Server 2014 will report a file type 0.)

type_desc

nvarchar(60)

Description of the file type:

ROWS (Includes files of full-text catalogs that are upgraded to or created in SQL Server 2014.)

LOG

FILESTREAM

FULLTEXT (Full-text catalogs earlier than SQL Server 2014.)

data_space_id

int

ID of the data space to which this file belongs. Data space is a filegroup.

0 = Log files

name

sysname

Logical name of the file in the database.

physical_name

nvarchar(260)

Operating-system file name.

state

tinyint

File state:

0 = ONLINE

1 = RESTORING

2 = RECOVERING

3 = RECOVERY_PENDING

4 = SUSPECT

5 = Identified for informational purposes only. Not supported. Future compatibility is not guaranteed.

6 = OFFLINE

7 = DEFUNCT

state_desc

nvarchar(60)

Description of the file state:

ONLINE

RESTORING

RECOVERING

RECOVERY_PENDING

SUSPECT

OFFLINE

DEFUNCT

For more information, see File States.

size

int

Current file size, in 8-KB pages. For a database snapshot, size reflects the maximum space that the snapshot can ever use for the file.

Note Note

This field is populated as zero for FILESTREAM containers. Query the sys.database_files catalog view for the actual size of FILESTREAM containers.

max_size

int

Maximum file size, in 8-KB pages:

0 = No growth is allowed.

-1 = File will grow until the disk is full.

268435456 = Log file will grow to a maximum size of 2 TB.

Note Note

Databases that are upgraded with an unlimited log file size will report -1 for the maximum size of the log file.

growth

int

0 = File is fixed size and will not grow.

>0 = File will grow automatically.

If is_percent_growth = 0, growth increment is in units of 8-KB pages, rounded to the nearest 64 KB

If is_percent_growth = 1, growth increment is expressed as a whole number percentage.

is_media_read_only

bit

1 = File is on read-only media.

0 = File is on read/write media.

is_read_only

bit

1 = File is marked read-only.

0 = file is marked read/write.

is_sparse

bit

1 = File is a sparse file.

0 = File is not a sparse file.

For more information, see View the Size of the Sparse File of a Database Snapshot (Transact-SQL).

is_percent_growth

bit

1 = Growth of the file is a percentage.

0 = Absolute growth size in pages.

is_name_reserved

bit

1 = Dropped file name is reusable. A log backup must be taken before the name (name or physical_name) can be reused for a new file name.

0 = File name is unavailable for reuse.

create_lsn

numeric(25,0)

Log sequence number (LSN) at which the file was created.

drop_lsn

numeric(25,0)

LSN at which the file was dropped.

read_only_lsn

numeric(25,0)

LSN at which the filegroup that contains the file changed from read/write to read-only (most recent change).

read_write_lsn

numeric(25,0)

LSN at which the filegroup that contains the file changed from read-only to read/write (most recent change).

differential_base_lsn

numeric(25,0)

Base for differential backups. Data extents changed after this LSN will be included in a differential backup.

differential_base_guid

uniqueidentifier

Unique identifier of the base backup on which a differential backup will be based.

differential_base_time

datetime

Time corresponding to differential_base_lsn.

redo_start_lsn

numeric(25,0)

LSN at which the next roll forward must start.

Is NULL unless state = RESTORING or state = RECOVERY_PENDING.

redo_start_fork_guid

uniqueidentifier

Unique identifier of the recovery fork. The first_fork_guid of the next log backup restored must match this value. This represents the current state of the container.

redo_target_lsn

numeric(25,0)

LSN at which the online roll forward on this file can stop.

Is NULL unless state = RESTORING or state = RECOVERY_PENDING.

redo_target_fork_guid

uniqueidentifier

The recovery fork on which the container can be recovered. Paired with redo_target_lsn.

backup_lsn

numeric(25,0)

The LSN of the most recent data or differential backup of the file.

Note Note

When you drop or rebuild large indexes, or drop or truncate large tables, the Database Engine defers the actual page deallocations, and their associated locks, until after the transaction commits. Deferred drop operations do not release allocated space immediately. Therefore, the values returned by sys.master_files immediately after dropping or truncating a large object may not reflect the actual disk space available.

Permissions

The minimum permissions that are required to see the corresponding row are CREATE DATABASE, ALTER ANY DATABASE, or VIEW ANY DEFINITION.

See Also

Reference

sys.all_views (SQL Server PDW)
sys.databases (SQL Server PDW)
sys.database_files (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server PDW Catalog Views (SQL Server PDW)


sys.numbered_procedures (SQL Server PDW)

[image]

sys.numbered_procedures (SQL Server PDW)
See Also Send Feedback

Contains a row for each SQL Server stored procedure that was created as a numbered procedure. This does not show a row for the base (number = 1) stored procedure. Entries for the base stored procedures can be found in views such as sys.objects and sys.procedures.

Important note Important

Numbered procedures are deprecated. Use of numbered procedures is discouraged. A DEPRECATION_ANNOUNCEMENT event is fired when a query that uses this catalog view is compiled.

Column name | Data type | Description
object_id | int | ID of the object of the stored procedure.
procedure_number | smallint | Number of this procedure within the object, 2 or greater.
definition | nvarchar(max) | The SQL Server text that defines this procedure. NULL = encrypted.
Note Note

XML and CLR parameters are not supported for numbered procedures.

Permissions

The visibility of the metadata in catalog views is limited to securables that a user either owns or on which the user has been granted some permission. For more information, see Metadata Visibility Configuration.

See Also

Reference

System Views (SQL Server PDW)


sys.objects (SQL Server PDW)

[image]

sys.objects (SQL Server PDW)
See Also Send Feedback

Contains a row for each user-defined object that is created within a database.

Column Name | Data Type | Description | Range
name | sysname | Object name. |
object_id | int | Object identification number. Is unique within a database. |
principal_id | int | id of the individual owner, if different from the schema owner. | See principal_id in sys.database_principals (SQL Server PDW).
schema_id | int | id of the schema that the object is contained in. Schema-scoped system objects are always contained in the sys or INFORMATION_SCHEMA schemas. |
parent_object_id | int | Id of the object to which this object belongs. 0 = Not a child object. |
type | char(2) | Object type: FN = SQL scalar function IT = Internal table P = SQL_STORED_PROCEDURE S = System base table U = Table (user-defined) V = View |
type_desc | nvarchar(60) | SQL_SCALAR_FUNCTION INTERNAL_TABLE SYSTEM_TABLE USER_TABLE VIEW |
create_date | datetime | Date the object was created. |
modify_date | datetime | Date the object was last modified by using an ALTER statement. If the object is a table or a view, modify_date also changes when an index on the table or view is created or altered. |
is_ms_shipped | bit | Object is created by an internal SQL Server component. |
is_published | bit | Object is published. | Always 0.
is_schema_published | bit | Only the schema of the object is published. | Always 0.

See Also

Reference

System Views (SQL Server PDW)
sys.all_objects (SQL Server PDW)


sys.partition_functions (SQL Server PDW)

[image]

sys.partition_functions (SQL Server PDW)
See Also Send Feedback

Contains a row for each partition function. SQL Server PDW does not support partition functions, but it does support this view for integration of tools and applications. This view will return 0 rows.

Column name | Data type | Description
name | sysname | Name of the partition function. Is unique within the database.
function_id | int | Partition function id. Is unique within the database.
type | char(2) | Function type. R = Range
type_desc | nvarchar(60) | Function type RANGE
fanout | int | Number of partitions created by the function.
boundary_value_on_right | bit | For range partitioning. 1 = Boundary value is included in the RIGHT range of the boundary. 0 = LEFT.
create_date | datetime | Date the function was created.
modify_date | datetime | Date the function was last modified using an ALTER statement.

See Also

Reference

System Views (SQL Server PDW)


sys.partition_parameters (SQL Server PDW)

[image]

sys.partition_parameters (SQL Server PDW)
See Also Send Feedback

Contains a row for each parameter of a partition function. SQL Server PDW does not support partition functions, but it does support this view for integration of tools and applications. This view will return 0 rows.

Column name | Data type | Description
function_id | int | ID of the partition function to which this parameter belongs.
parameter_id | int | ID of the parameter. Is unique within the partition function, beginning with 1.
system_type_id | tinyint | ID of the system type of the parameter. Corresponds to the system_type_id column of the sys.types catalog view.
max_length | smallint | Maximum length of the parameter in bytes.
precision | tinyint | Precision of the parameter if numeric-based; otherwise, 0.
scale | tinyint | Scale of the parameter if numeric-based; otherwise, 0.
collation_name | sysname | Name of the collation of the parameter if character-based; otherwise, NULL.
user_type_id | int | ID of the type. Is unique within the database. For system data types, user_type_id = system_type_id.

See Also

Reference

System Views (SQL Server PDW)


sys.partition_range_values (SQL Server PDW)

[image]

sys.partition_range_values (SQL Server PDW)
See Also Send Feedback

Contains a row for each range boundary value of a partition function of type R. SQL Server PDW does not support partition functions, but it does support this view for integration of tools and applications. This view will return 0 rows.

Column name | Data type | Description
function_id | int | id of the partition function for this range boundary value.
boundary_id | int | id (1-based ordinal) of the boundary value tuple, with left-most boundary starting at an id of 1.
parameter_id | int | ID of the parameter of the function to which this value corresponds. The values in this column correspond with those in the parameter_id column of the sys.partition_parameters catalog view for any particular function_id.
value | sql_variant | The actual boundary value.

See Also

Reference

System Views (SQL Server PDW)


sys.partition_schemes (SQL Server PDW)

[image]

sys.partition_schemes (SQL Server PDW)
See Also Send Feedback

Contains a row for each Data Space that is a partition scheme, with type = PS. SQL Server PDW does not support partition schemes, but it does support this view for integration of tools and applications. This view will return 0 rows.

Column name | Data type | Description
<inherited columns> |  | Inherits columns from sys.data_spaces (SQL Server PDW).
function_id | int | ID of partition function used in the scheme.

See Also

Reference

System Views (SQL Server PDW)


sys.partitions (SQL Server PDW)

[image]

sys.partitions (SQL Server PDW)
See Also Send Feedback

Contains a row for each partition of all the tables, and most types of indexes in a SQL Server PDWdatabase. All tables and indexes contain at least one partition, whether or not they are explicitly partitioned.

Column name

Data type

Description

partition_id

bigint

id of the partition. Is unique within a database.

object_id

int

id of the object to which this partition belongs. Every table or view is composed of at least one partition.

index_id

int

id of the index within the object to which this partition belongs.

partition_number

int

1-based partition number within the owning index or heap. For SQL Server PDW, the value of this column is 1.

hobt_id

bigint

ID of the data heap or B-tree that contains the rows for this partition.

rows

bigint

Approximate average number of rows in each table partition. To calculate this value, SQL Server PDW divides the number of rows in the table by the number of partitions in the table.

SQL Server PDW uses statistics, which might be out-of-date, to determine the total number of rows. The statistics are from the most recent run of UPDATE STATISTICS on the table. If UPDATE STATISTICS has not been run on the table, the statistics won’t exist, and SQL Server PDW will use 1000 as the default total number of rows.

To display the number of rows in each partition within each distribution, use DBCC PDW_SHOWPARTITIONSTATS (SQL Server PDW).

data_compression

int

Indicates the state of compression for each partition:

0 = NONE

1 = ROW

2 = PAGE

3 = COLUMNSTORE

data_compression_desc

nvarchar(60)

Indicates the state of compression for each partition. Possible values are NONE, ROW, and PAGE.

See Also

Reference

System Views (SQL Server PDW)


sys.parameters (SQL Server PDW)

[image]

sys.parameters (SQL Server PDW)
See Also Send Feedback

Contains a row for each parameter of a SQL Server PDW object that accepts parameters.

Column name | Data type | Description
object_id | int | id of the object to which this parameter belongs.
name | sysname | Name of the parameter. Is unique within the object. If the object is a scalar function, the parameter name is an empty string in the row representing the return value.
parameter_id | int | id of the parameter. Is unique within the object. If the object is a scalar function, parameter_id = 0 represents the return value.
system_type_id | tinyint | id of the system type of the parameter.
user_type_id | int | id of the parameter type. To return the name of the type, join to the sys.types (SQL Server PDW) catalog view on this column.
max_length | smallint | Maximum length of the parameter, in bytes. Value = -1 when the column data type is varchar(max), nvarchar(max), varbinary(max), or xml.
precision | tinyint | Precision of the parameter if numeric-based; otherwise, 0.
scale | tinyint | Scale of the parameter if numeric-based; otherwise, 0.
is_output | bit | 1 = Parameter is RETURN; otherwise, 0.
is_cursor_ref | bit | Always 0.
has_default_value | bit | Always 0.
is_xml_document | bit | Always 0.
default_value | sql_variant | Always NULL.
xml_collection_id | int | Always 0.
is_readonly | int | Always 0.

Permissions

Users can see information about objects for which they have some type of permission.

See Also

Reference

System Views (SQL Server PDW)
CREATE PROCEDURE (SQL Server PDW)


sys.procedures (SQL Server PDW)

[image]

sys.procedures (SQL Server PDW)
See Also Send Feedback

Contains a row for every stored procedure in SQL Server PDW.

Column name | Data type | Description
<Columns inherited from sys.objects> |  | For a list of columns that this view inherits, see sys.objects (SQL Server PDW)
is_auto_executed | bit | Always 0.
is_execution_replicated | bit | Always 0.
is_repl_serializable_only | bit | Always 0.
skips_repl_constraints | bit | Always 0.

Permissions

Users can see information about objects for which they have some type of permission.

See Also

Reference

CREATE PROCEDURE (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


sys.schemas (SQL Server PDW)

[image]

sys.schemas (SQL Server PDW)
See Also Send Feedback

Contains a row for each database schema.

Column Name | Data Type | Description | Range
name | sysname | Name of the schema. | Built-in schemas are: 'dbo','INFORMATION_SCHEMA','sys', 'sysdiag.' User-defined schemas are available beginning with APS AU2.
schema_id | int | id of the schema. | 1= dbo, 3 = INFORMATION_SCHEMA, 4 = sys, 5=sysdiag
principal_id | int | id of principal that owns this schema. | See principal_id in sys.database_principals (SQL Server PDW).

See Also

Reference

System Views (SQL Server PDW)
CREATE SCHEMA (SQL Server PDW)
ALTER SCHEMA (SQL Server PDW)
sys.schemas (SQL Server PDW)


sys.securable_classes (SQL Server PDW)

[image]

sys.securable_classes (SQL Server PDW)
See Also Send Feedback

Returns a list of securable classes supported by SQL Server PDW. For more information, see sys.securable_classes (Transact-SQL).

See Also

Reference

System Views (SQL Server PDW)


sys.server_role_members (SQL Server PDW)

[image]

sys.server_role_members (SQL Server PDW)
See Also Send Feedback

Returns one row for each member of each fixed server role in SQL Server PDW.

Column name | Data type | Description
role_principal_id | int | Server-Principal ID of the role.
member_principal_id | int | Server-Principal ID of the member.

Permissions

Logins can view their own server role membership and can view the principal_id’s of the members of the fixed server roles. To view all server role membership requires the VIEW DEFINITION ON SERVER ROLE.

Examples

The following query returns the members of the server roles.

Copy image Copy Code
SELECT SP1.name AS ServerRoleName,
 isnull (SP2.name, 'No members') AS LoginName
 FROM sys.server_role_members AS SRM
 RIGHT OUTER JOIN sys.server_principals AS SP1
   ON SRM.role_principal_id = SP1.principal_id
 LEFT OUTER JOIN sys.server_principals AS SP2
   ON SRM.member_principal_id = SP2.principal_id
 ORDER BY SP1.name;

See Also

Reference

System Views (SQL Server PDW)


sys.server_permissions (SQL Server PDW)

[image]

sys.server_permissions (SQL Server PDW)
See Also Send Feedback

Returns one row for each server-level permission in SQL Server PDW.

Note Note

Fixed server roles are not available in this release. Assign permissions to user-defined database roles instead.

Column name

Data type

Description

class

tinyint

Identifies class of thing on which permission exists.

100 = Server

101 = Server-principal

105 = Endpoint (Not available in PDW)

class_desc

nvarchar(60)

Description of class on which permission exists.

  • SERVER

  • SERVER_PRINCIPAL

  • ENDPOINT (Not available in PDW)

major_id

int

id of the securable on which permission exists, interpreted according to class. For most, this is just the kind of id that applies to what the class represents. Interpretation for non-standard is as follows:

100 = Always 0

minor_id

int

Secondary id of thing on which permission exists, interpreted according to class.

grantee_principal_id

int

Server-principal-id to which the permissions are granted.

grantor_principal_id

int

Server-principal-id of the grantor of these permissions.

type

char(4)

Server permission type. For a list of permission types, see the next table.

permission_name

nvarchar(128)

Permission name.

state

char(1)

Permission state:

D = Deny

R = Revoke

G = Grant

W = Grant With Grant option

state_desc

nvarchar(60)

Description of permission state:

DENY

REVOKE

GRANT

GRANT_WITH_GRANT_OPTION

Permission type | Permission name | Applies to securable
ADBO | ADMINISTER BULK OPERATIONS | SERVER
AL | ALTER | LOGIN
ALCO | ALTER ANY CONNECTION | SERVER
ALDB | ALTER ANY DATABASE | SERVER
ALES | ALTER ANY EVENT NOTIFICATION | SERVER
ALLG | ALTER ANY LOGIN | SERVER
ALSS | ALTER SERVER STATE | SERVER
CL | CONTROL | LOGIN
CL | CONTROL SERVER | SERVER
COSQ | CONNECT SQL | SERVER
CRDB | CREATE ANY DATABASE | SERVER
VW | VIEW DEFINITION | LOGIN
VWAD | VIEW ANY DEFINITION | SERVER
VWDB | VIEW ANY DATABASE | SERVER
VWSS | VIEW SERVER STATE | SERVER

Permissions

Any user can see their own permissions. To see permissions for other logins, requires VIEW DEFINITION, ALTER ANY LOGIN, or any permission on a login.

Examples

The following query lists the permissions explicitly granted or denied to server principals.

Important note Important

The permissions of fixed server roles do not appear in sys.server_permissions. Therefore, server principals may have additional permissions not listed here.

Copy image Copy Code
SELECT pr.principal_id, pr.name, pr.type_desc,
    pe.state_desc, pe.permission_name
FROM sys.server_principals AS pr
JOIN sys.server_permissions AS pe
    ON pe.grantee_principal_id = pr.principal_id;

See Also

Reference

System Views (SQL Server PDW)
sys.database_permissions (SQL Server PDW)
sys.server_principals (SQL Server PDW)


sys.server_principals (SQL Server PDW)

[image]

sys.server_principals (SQL Server PDW)
See Also Send Feedback

Contains a row for every server-level principal in SQL Server PDW.

Column name | Data type | Description
name | sysname | Name of the principal. Is unique within a server.
principal_id | int | ID number of the principal. Is unique within a server.
sid | varbinary(85) | SID (Security-Identifier) of the principal. If Windows principal, then it matches Windows SID.
type | char(1) | Principal type: S = SQL login U = Windows login G = Windows group R = Server role C = Login mapped to a certificate (Not available in PDW) K = Login mapped to an asymmetric key (Not available in PDW)
type_desc | nvarchar(60) | Description of the principal type: SQL_LOGIN WINDOWS_LOGIN WINDOWS_GROUP SERVER_ROLE CERTIFICATE_MAPPED_LOGIN ASYMMETRIC_KEY_MAPPED_LOGIN
is_disabled | int | 1 = Login is disabled.
create_date | datetime | Time at which the principal was created.
modify_date | datetime | Time at which the principal definition was last modified.
default_database_name | sysname | Default database for this principal.
default_language_name | sysname | Default language for this principal.
credential_id | int | ID of a credential associated with this principal. If no credential is associated with this principal, credential_id will be NULL.
owning_principal_id |  | The principal_id of the owner of a server role. NULL if the principal is not a server role.
is_fixed_role |  | Returns 1 if the principal is one of the fixed server roles.

Permissions

Any login can see their own login name, the system logins, and the fixed server roles. To see other logins, requires ALTER ANY LOGIN, or a permission on the login.

Examples

The following query lists the permissions explicitly granted or denied to server principals.

Important note Important

The permissions of fixed server roles do not appear in sys.server_permissions. Therefore, server principals may have additional permissions not listed here.

Copy image Copy Code
SELECT pr.principal_id, pr.name, pr.type_desc,
    pe.state_desc, pe.permission_name
FROM sys.server_principals AS pr
JOIN sys.server_permissions AS pe
    ON pe.grantee_principal_id = pr.principal_id;

See Also

Reference

System Views (SQL Server PDW)
sys.database_principals (SQL Server PDW)


sys.sql_expression_dependencies (SQL Server PDW)

[image]

sys.sql_expression_dependencies (SQL Server PDW)
See Also Send Feedback

Contains one row for each by-name dependency on a user-defined entity in the current database. A dependency between two entities is created when one entity (called the reference entity) appears by name in a persisted SQL expression of another entity (called the referencing entity). For example, when a table is referenced in the definition of a view, the view (as the referencing entity) depends on the table (the referenced entity). If the table is dropped, the view is unusable.

Column Name

Data Type

Description

Range

referencing_id

int

ID of the referencing entity. Is not nullable.

referencing_minor_id

int

Column ID when the referencing entity is a column; otherwise 0. Is not nullable.

referencing_class

tinyint

Class of the referencing entity.

1 = Object or column

Always 1.

referencing_class_desc

nvarchar(60)

Description of the class of referencing entity.

OBJECT_OR_COLUMN

is_schema_bound_reference

bit

1 = Referenced entity is schema-bound.

0 = Referenced entity is non-schema-bound.

referenced_class

tinyint

Class of the referenced entity.

1 = Object or column

6 = Type

10 = XML schema collection

21 = Partition function

referenced_class_desc

nvarchar(60)

Description of class of referenced entity.

OBJECT_OR_COLUMN

TYPE

XML_SCHEMA_COLLECTION

PARTITION_FUNCTION

referenced_server_name

sysname

Always NULL.

referenced_database_name

sysname

Name of the database of the referenced entity.

NULL for non-schema-bound references when specified using a one-part or two-part name.

NULL for schema-bound entities because they must be in the same database and therefore can only be defined using a two-part (schema.object) name.

referenced_schema_name

sysname

Schema in which the referenced entity belongs.

NULL for non-schema-bound references in which the entity was referenced without specifying the schema name.

Never NULL for schema-bound references because schema-bound entities must be defined and referenced by using a two-part name.

referenced_entity_name

sysname

Name of the referenced entity. Is not nullable.

referenced_id

int

ID of the referenced entity.

NULL for references within the database if the ID cannot be determined. For non-schema-bound references, the ID cannot be resolved in the following cases:

The referenced entity does not exist in the database.

The schema of the referenced entity depends on the schema of the caller and is resolved at run time. In this case, is_caller_dependent is set to 1.

Never NULL for schema-bound references.

referenced_minor_id

int

ID of the referenced column when the referencing entity is a column; otherwise 0. Is not nullable. A referenced entity is a column when a column is identified by name in the referencing entity, or when the parent entity is used in a SELECT * statement.

is_caller_dependent

bit

Always 0.

is_ambiguous

bit

1 = Reference is ambiguous.

0 = Reference is unambiguous or the entity can be successfully bound when the view is called.

Always 0 for schema bound references.

Always 0.

See Also

Reference

System Views (SQL Server PDW)


sys.sql_logins (SQL Server PDW)

[image]

sys.sql_logins (SQL Server PDW)
See Also Send Feedback

Returns one row for every SQL Server authentication login.

Column name | Data type | Description
<inherited columns> | -- | Inherits from sys.server_principals.
is_policy_checked | bit | Password policy is checked.
is_expiration_checked | bit | Password expiration is checked.
password_hash | varbinary(256) | Hash of the SQL Server authentication login password.

For a list of columns that this view inherits, see sys.server_principals (SQL Server PDW).

Permissions

Any SQL Server authentication login can see their own login name. To see other logins, requires ALTER ANY LOGIN, or a permission on the login.

See Also

Reference

System Views (SQL Server PDW)
sys.server_principals (SQL Server PDW)
sys.database_principals (SQL Server PDW)


sys.sql_modules (SQL Server PDW)

[image]

sys.sql_modules (SQL Server PDW)
See Also Send Feedback

Returns SQL Server PDW views and stored procedures.

Column Name | Data Type | Description | Range
object_id | int | ID of the object of the containing object. Is unique within a database. |
definition | nvarchar(max) | SQL text that defines this module. |
uses_ansi_nulls | bit | Module was created with SET ANSI_NULLS ON. Will always be = 0 for rules and defaults. |
uses_quoted_identifier | bit | Module was created with SET QUOTED_IDENTIFIER ON. |
is_schema_bound | bit | Module was created with SCHEMABINDING option. |
uses_database_collation | bit |  | Always 0.
is_recompiled | bit | Procedure was created WITH RECOMPILE option. | Always 0.
null_on_null_input | bit | Module was declared to produce a NULL output on any NULL input. | Always 0.
execute_as_principal_id | int |  | Always NULL.

See Also

Reference

System Views (SQL Server PDW)
sys.all_sql_modules (SQL Server PDW)


sys.stats (SQL Server PDW)

[image]

sys.stats (SQL Server PDW)
See Also Send Feedback

Contains a row for each statistic of a tabular object of the type user-defined table or view. Every index will have a corresponding statistics row with the same name and ID (index_id=stats_id), but not every statistics row has a corresponding index.

Column Name | Data Type | Description | Range
object_id | int | ID of the object to which these statistics belong. |
name | sysname | Name of the statistic. Is unique within the object. |
stats_id | int | ID of the statistics. Is unique within the object. |
auto_created | bit | Statistics were auto-created by the query processor. |
user_created | bit | Statistics were explicitly created by the user. |
no_recompute | bit | Statistics were created with the NORECOMPUTE option. |
has_filter | bit | 1 = Statistics have a filter and are computed only on rows that satisfy the filter definition. |
filter_definition | nvarchar(max) | Expression for the subset of rows included in filtered statistics. NULL = Non-filtered statistics. |

See Also

Reference

System Views (SQL Server PDW)


sys.stats_columns (SQL Server PDW)

[image]

sys.stats_columns (SQL Server PDW)
See Also Send Feedback

Contains a row for each column that is part of sys.stats (SQL Server PDW) statistics.

Column Name | Data Type | Description | Range
object_id | int | id of the object of which this column is part. |
stats_id | int | id of the statistics of which this column is part. |
stats_column_id | bit | 1-based ordinal within set of statistics columns. |
column_id | bit | id of the column from sys.columns (SQL Server PDW). |

See Also

Reference

System Views (SQL Server PDW)


sys.system_columns (SQL Server PDW)

[image]

sys.system_columns (SQL Server PDW)
See Also Send Feedback

Shows columns for system objects.

Column Name | Data Type | Description | Range
object_id | int | ID of the object to which this column belongs. |
name | sysname | Name of the column. Unique in object. |
column_id | int | ID of the column. Unique in object. |
system_type_id | tinyint | ID of the system type of the column. |
user_type_id | int | ID of the type of the column as defined by the user. |
max_length | smallint | Maximum length (in bytes) of the column. | (-1) not valid
precision | tinyint | Precision of the column if numeric-based; otherwise, 0. |
scale | tinyint | Scale of column if numeric-based; otherwise, 0. |
collation_name | sysname | Name of the collation of the column if character-based; otherwise, NULL. | Latin1_General_100_CI_AS_KS_WS
is_nullable | bit | 1 = Column is nullable. |
is_ansi_padded | bit | 1 = Column uses ANSI_PADDING ON behavior if character, binary, or variant. | Always 0 .
is_rowguidcol | bit | 1 = Column is a declared ROWGUIDCOL. | Always 0 .
is_identity | bit | 1 = Column has identity values. | Always 0 .
is_computed | bit | 1 = Column is a computed column. | Always 0 .
is_filestream | bit | 1 = Column is a FILESTREAM column. | Always 0 .
is_replicated | bit | 1 = Column is replicated. | Always 0 .
is_non_sql_subscribed | bit | 1 = Column has a non-SQL subscriber. | Always 0 .
is_merge_published | bit | 1 = Column is merge-published. | Always 0 .
is_dts_replicated | bit | 1 = Column is replicated by using SSIS. | Always 0 .
is_xml_document | bit | 1 = Content is a complete XML document. | Always 0 .
xml_collection_id | int | 0 = No XML schema collection. | Always 0 .
default_object_id | int | ID of the default object. 0 = No default. | Always 0 .
rule_object_id | int | ID of the stand-alone rule bound to the column. 0 = No stand-alone rule. | Always 0 .
is_sparse | bit | 1 = Column is a sparse column. | Always 0 .
is_column_set | bit | 1 = Column is a column set. | Always 0 .
distribution_ordinal | tinyint | Ordinal (1-based) within a distribution set. 0 = Not a distribution column. | 1 = This column is used to distribute the table this column belongs to. 0 = This column is not used for distribution.

See Also

Reference

System Views (SQL Server PDW)


sys.system_objects (SQL Server PDW)

[image]

sys.system_objects (SQL Server PDW)
See Also Send Feedback

Returns system objects, including tables and views.

Column Name | Data Type | Description | Range
name | sysname | Object name. |
object_id | int | Object identification number. Is unique within a database. |
principal_id | int | id of the individual owner, if different from the schema owner. | See principal_id in sys.database_principals (SQL Server PDW).
schema_id | int | id of the schema that the object is contained in. Schema-scoped system objects are always contained in the sys or INFORMATION_SCHEMA schemas. |
parent_object_id | int | id of the object to which this object belongs. 0 = Not a child object. |
type | char(2) | Object type: FN = SQL scalar function IT = Internal table S = System base table U = Table (user-defined) V = View | FN, IT, S, U, V
type_desc | nvarchar(60) | SQL_SCALAR_FUNCTION INTERNAL_TABLE SYSTEM_TABLE USER_TABLE VIEW | SQL_SCALAR_FUNCTION, INTERNAL_TABLE, SYSTEM_TABLE, USER_TABLE, VIEW
create_date | datetime | Date the object was created. |
modify_date | datetime | Date the object was last modified by using an ALTER statement. If the object is a table or a view, modify_date also changes when a clustered index on the table or view is created or altered. |
is_ms_shipped | bit | Object is created by an internal SQL Server component. |
is_published | bit | Object is published. | Always 0.
is_schema_published | bit | Only the schema of the object is published. | Always 0.

See Also

Reference

System Views (SQL Server PDW)


sys.system_sql_modules (SQL Server PDW)

[image]

sys.system_sql_modules (SQL Server PDW)
See Also Send Feedback

Returns SQL Server system modules.

Column Name | Data Type | Description | Range
object_id | int | ID of the object of the containing object. Is unique within a database. |
definition | nvarchar(max) | SQL text that defines this module. |
uses_ansi_nulls | bit | Module was created with SET ANSI_NULLS ON. Will always be = 0 for rules and defaults. |
uses_quoted_identifier | bit | Module was created with SET QUOTED_IDENTIFIER ON. |
is_schema_bound | bit | Module was created with SCHEMABINDING option. |
uses_database_collation | bit |  | Always 0.
is_recompiled | bit | Procedure was created WITH RECOMPILE option. | Always 0.
null_on_null_input | bit | Module was declared to produce a NULL output on any NULL input. | Always 0.
execute_as_principal_id | int |  | Always NULL.

See Also

Reference

System Views (SQL Server PDW)


sys.system_views (SQL Server PDW)

[image]

sys.system_views (SQL Server PDW)
See Also Send Feedback

Same definition as sys.views, but shows system views only.

Column Name | Data Type | Description | Range
<inherited columns from sys.objects> |  | See sys.objects (SQL Server PDW) |
is_replicated | bit | 1 = View is replicated. | Always 0.
has_replication_filter | bit | 1 = View has a replication filter. | Always 0.
has_opaque_metadata | bit | 1 = VIEW_METADATA option specified for view. | Always 0.
has_unchecked_assembly_data | bit | Information not available. | Always 0, no CLR support.
with_check_option | bit | 1 = WITH CHECK OPTION was specified in the view definition. | Always 0.
is_date_correlation_view | bit | 1 = View was created automatically by the system to store correlation information between datetime columns. | Always 0.
is_tracked_by_cdc | bit | Information not available. | Always 0.

See Also

Reference

System Views (SQL Server PDW)


sys.tables (SQL Server PDW)

[image]

sys.tables (SQL Server PDW)
See Also Send Feedback

Contains a row for each table object that a principal either owns or on which the principal has been granted some permission.

Column Name | Data Type | Description | Range
<inherited columns> |  | For a list of columns that this view inherits, see sys.objects. |
lob_data_space_id | int |  | Always 0.
filestream_data_space_id | int | Data space ID for a FILESTREAM filegroup or Information not available. | NULL
max_column_id_used | int | Maximum column ID used by this table. |
lock_on_bulk_load | bit | Table is locked on bulk load. | TBD
uses_ansi_nulls | bit | Table was created with the SET ANSI_NULLS database option ON. | 1
is_replicated | bit | 1 = Table is published using replication. | 0; replication is not supported.
has_replication_filter | bit | 1 = Table has a replication filter. | 0
is_merge_published | bit | 1 = Table is published using merge replication. | 0; not supported.
is_sync_tran_subscribed | bit | 1 = Table is subscribed using an immediate updating subscription. | 0; not supported.
has_unchecked_assembly_data | bit | 1 = Table contains persisted data that depends on an assembly whose definition changed during the last ALTER ASSEMBLY. Will be reset to 0 after the next successful DBCC CHECKDB or DBCC CHECKTABLE. | 0; no CLR support.
text_in_row_limit | int | 0 = Text in row option is not set. | Always 0.
large_value_types_out_of_row | bit | 1 = Large value types are stored out-of-row. | Always 0.
is_tracked_by_cdc | bit | 1 = Table is enabled for change data capture | Always 0; no CDC support.
lock_escalation | tinyint | The value of the LOCK_ESCALATION option for the table: 2 = AUTO | Always 2.
lock_escalation_desc | nvarchar(60) | A text description of the lock_escalation option. | Always ꞌAUTOꞌ.

See Also

Reference

System Views (SQL Server PDW)


sys.types (SQL Server PDW)

[image]

sys.types (SQL Server PDW)
See Also Send Feedback

Contains a row for each system type.

Column Name | Data Type | Description | Range
name | sysname | Name of the type. Is unique within the schema. |
system_type_id | tinyint | ID of the internal system-type of the type. |
user_type_id | int | ID of the type. Is unique within the database. For system data types, see user_type_id = system_type_id. |
schema_id | int | ID of the schema to which the type belongs. |
principal_id | int | ID of the individual owner if different from schema owner. By default, schema-contained objects are owned by the schema owner. However, an alternate owner can be specified by using the ALTER AUTHORIZATION statement to change ownership. NULL if there is no alternate individual owner. |
max_length | smallint | Maximum length (in bytes) of the type. |
precision | tinyint | Maximum precision of the type if it is numeric-based; otherwise, 0. |
scale | tinyint | Maximum scale of the type if it is numeric-based; otherwise, 0. |
collation_name | sysname | Name of the collation of the type if it is character-based; otherwise, NULL. |
is_nullable | bit | Type is nullable. |
is_user_defined | bit | 1 = User-defined type 0 = SQL Server system data type | Always 0.
is_assembly_type | bit | 1 = Implementation of the type is defined in a CLR assembly. | Always 0.
default_object_id | int | 0 = No default exists. | Always 0.
rule_object_id | int | 0 = No rule exists. | Always 0.
is_table_type | bit | Indicates that the type is a table. | Always 0.

See Also

Reference

System Views (SQL Server PDW)


sys.views (SQL Server PDW)

[image]

sys.views (SQL Server PDW)
See Also Send Feedback

Contains a row for each view object. The visibility of table objects is limited to securables that a user either owns or on which the user has been granted some permission.

Column Name | Data Type | Description | Range
<inherited columns from sys.objects> |  | See sys.objects (SQL Server PDW) |
is_replicated | bit | 1 = View is replicated. | Always 0.
has_replication_filter | bit | 1 = View has a replication filter. | Always 0.
has_opaque_metadata | bit | 1 = VIEW_METADATA option specified for view. | Always 0.
has_unchecked_assembly_data | bit | Information not available. | Always 0, no CLR support.
with_check_option | bit | 1 = WITH CHECK OPTION was specified in the view definition. | Always 0.
is_date_correlation_view | bit | 1 = View was created automatically by the system to store correlation information between datetime columns. | Always 0.
is_tracked_by_cdc | bit | Information not available. | Always 0.

See Also

Reference

sys.all_views (SQL Server PDW)

Concepts

Common Metadata Query Examples (SQL Server PDW)
SQL Server Catalog Views (SQL Server PDW)


SQL Server Compatibility Views (SQL Server PDW)

[image]

SQL Server Compatibility Views (SQL Server PDW)
See Also Send Feedback

Lists the SQL Server compatibility views available in SQL Server PDW. These are catalog views available in SQL Server 2000 that continue to be used by tools.

List of SQL Server Compatibility Views Available in SQL Server PDW

Catalog View Name | Link to SQL Server Transact-SQL topic on MSDN | Results
sys.sysdatabases | sys.sysdatabases (Transact-SQL) | Same results as sys.databases (SQL Server PDW).
sys.sysobjects | sys.sysobjects (Transact-SQL) | Same results as sys.objects (SQL Server PDW).
sys.systypes | sys.systypes (Transact-SQL) | Same results as sys.types (SQL Server PDW).
sys.sysusers | sys.sysusers (Transact-SQL) | Same results as sys.database_principals (SQL Server PDW).

See Also


SQL Server Dynamic Management Views (SQL Server PDW)

[image]

SQL Server Dynamic Management Views (SQL Server PDW)
See Also Send Feedback

Lists the SQL Server Dynamic Management Views (DMVs) available in SQL Server PDW.

List of SQL Server DMVs Available in SQL Server PDW

SQL Server PDW exposes many of the SQL Server dynamic management views (DMVs). These views, when queried in SQL Server PDW, are reporting the state of SQL Server running on the Compute Nodes.

Each SQL Server DMV has a PDW-specific column called pdw_node_id, which is the identifier for the Compute node.

Note Note

To use SQL Server PDW DMVs in SQL Server PDW, insert ‘pdw_nodes_’ into the name, as shown in the following table.

DMV Name in SQL Server PDW | Link to SQL Server Transact-SQL topic on MSDN
sys.dm_pdw_nodes_db_file_space_usage | sys.dm_db_file_space_usage (Transact-SQL)
sys.dm_pdw_nodes_db_index_usage_stats | sys.dm_db_index_usage_stats (Transact-SQL)
sys.dm_pdw_nodes_db_partition_stats | sys.dm_db_partition_stats (Transact-SQL)
sys.dm_pdw_nodes_db_session_space_usage | sys.dm_db_session_space_usage (Transact-SQL)
sys.dm_pdw_nodes_db_task_space_usage | sys.dm_db_task_space_usage (Transact-SQL)
sys.dm_pdw_nodes_exec_background_job_queue | sys.dm_exec_background_job_queue (Transact-SQL)
sys.dm_pdw_nodes_exec_background_job_queue_stats | sys.dm_exec_background_job_queue_stats (Transact-SQL)
sys.dm_pdw_nodes_exec_cached_plans | sys.dm_exec_cached_plans (Transact-SQL)
sys.dm_pdw_nodes_exec_connections | sys.dm_exec_connections (Transact-SQL)
sys.dm_pdw_nodes_exec_procedure_stats | sys.dm_exec_procedure_stats (Transact-SQL)
sys.dm_pdw_nodes_exec_query_memory_grants | sys.dm_exec_query_memory_grants (Transact-SQL)
sys.dm_pdw_nodes_exec_query_optimizer_info | sys.dm_exec_query_optimizer_info (Transact-SQL)
sys.dm_pdw_nodes_exec_query_resource_semaphores | sys.dm_exec_query_resource_semaphores (Transact-SQL)
sys.dm_pdw_nodes_exec_query_stats | sys.dm_exec_query_stats (Transact-SQL)
sys.dm_pdw_nodes_exec_requests | sys.dm_exec_requests (Transact-SQL)
sys.dm_pdw_nodes_exec_sessions | sys.dm_exec_sessions (Transact-SQL)
sys.dm_pdw_nodes_io_cluster_shared_drives | sys.dm_io_cluster_shared_drives (Transact-SQL)
sys.dm_pdw_nodes_io_pending_io_requests | sys.dm_io_pending_io_requests (Transact-SQL)
sys.dm_pdw_nodes_os_buffer_descriptors | sys.dm_os_buffer_descriptors (Transact-SQL)
sys.dm_pdw_nodes_os_child_instances | sys.dm_os_child_instances (Transact-SQL)
sys.dm_pdw_nodes_os_cluster_nodes | sys.dm_os_cluster_nodes (Transact-SQL)
sys.dm_pdw_nodes_os_dispatcher_pools | sys.dm_os_dispatcher_pools (Transact-SQL)
sys.dm_pdw_nodes_os_dispatchers | Transact-SQL documentation is not available.
sys.dm_pdw_nodes_os_hosts | sys.dm_os_hosts (Transact-SQL)
sys.dm_pdw_nodes_os_latch_stats | sys.dm_os_latch_stats (Transact-SQL)
sys.dm_pdw_nodes_os_loaded_modules | sys.dm_os_loaded_modules (Transact-SQL)
sys.dm_pdw_nodes_os_memory_brokers | sys.dm_os_memory_brokers (Transact-SQL)
sys.dm_pdw_nodes_os_memory_cache_clock_hands | sys.dm_os_memory_cache_clock_hands (Transact-SQL)
sys.dm_pdw_nodes_os_memory_cache_counters | sys.dm_os_memory_cache_counters (Transact-SQL)
sys.dm_pdw_nodes_os_memory_cache_entries | sys.dm_os_memory_cache_entries (Transact-SQL)
sys.dm_pdw_nodes_os_memory_cache_hash_tables | sys.dm_os_memory_cache_hash_tables (Transact-SQL)
sys.dm_pdw_nodes_os_memory_clerks | sys.dm_os_memory_clerks (Transact-SQL)
sys.dm_pdw_nodes_os_memory_node_access_stats | Transact-SQL documentation is not available.
sys.dm_pdw_nodes_os_memory_nodes | sys.dm_os_memory_nodes (Transact-SQL)
sys.dm_pdw_nodes_os_memory_objects | sys.dm_os_memory_objects (Transact-SQL)
sys.dm_pdw_nodes_os_memory_pools | sys.dm_os_memory_pools (Transact-SQL)
sys.dm_pdw_nodes_os_nodes | sys.dm_os_nodes (Transact-SQL)
sys.dm_pdw_nodes_os_performance_counters | sys.dm_os_performance_counters (Transact-SQL)
sys.dm_pdw_nodes_os_process_memory | sys.dm_os_process_memory (Transact-SQL)
sys.dm_pdw_nodes_os_schedulers | sys.dm_os_schedulers (Transact-SQL)
sys.dm_pdw_nodes_os_spinlock_stats | Transact-SQL documentation is not available.
sys.dm_pdw_nodes_os_sys_info | sys.dm_os_sys_info (Transact-SQL)
sys.dm_pdw_nodes_os_sys_memory | sys.dm_os_memory_nodes (Transact-SQL)
sys.dm_pdw_nodes_os_tasks | sys.dm_os_tasks (Transact-SQL)
sys.dm_pdw_nodes_os_threads | sys.dm_os_threads (Transact-SQL)
sys.dm_pdw_nodes_os_virtual_address_dump | sys.dm_virtual_address_dump (Transact-SQL)
sys.dm_pdw_nodes_os_wait_stats | sys.dm_os_wait_stats (Transact-SQL)
sys.dm_pdw_nodes_os_waiting_tasks | sys.dm_waiting_tasks (Transact-SQL)
sys.dm_pdw_nodes_os_workers | sys.dm_os_workers (Transact-SQL)
sys.dm_pdw_nodes_resource_governor_resource_pools | sys.dm_ resource_governor_resource_pools
sys.dm_pdw_nodes_resource_governor_workload_groups | sys.dm_resource_governor_workload_groups
sys.dm_pdw_nodes_tran_active_snapshot_database_transactions | sys.dm_tran_active_snapshot_database_transactions (Transact-SQL)
sys.dm_pdw_nodes_tran_active_transactions | sys.dm_tran_active_transactions (Transact-SQL)
sys.dm_pdw_nodes_tran_commit_table | Transact-SQL documentation is not available.
sys.dm_pdw_nodes_tran_current_snapshot | sys.dm_tran_current_snapshot (Transact-SQL)
sys.dm_pdw_nodes_tran_database_transactions | sys.dm_tran_database_transactions (Transact-SQL)
sys.dm_pdw_nodes_tran_locks | sys.dm_tran_locks (Transact-SQL)
sys.dm_pdw_nodes_tran_session_transactions | sys.dm_tran_session_transactions (Transact-SQL)
sys.dm_pdw_nodes_tran_top_version_generators | sys.dm_tran_top_version_generators (Transact-SQL)

See Also

Reference

System Views (SQL Server PDW)


SQL Server Information_Schema Views (SQL Server PDW)

[image]

SQL Server Information_Schema Views (SQL Server PDW)
See Also Send Feedback

This topic links to the SQL Server Information Schema views available in SQL Server PDW. All of the INFORMATION_SCHEMA views that you would expect to see in SQL Server are available in SQL Server PDW.

Information Schema Views

The following links point to the SQL Server documentation on MSDN.

CHECK_ CONSTRAINTS (Transact-SQL

COLUMN_DOMAIN_USAGE (Transact-SQL)

COLUMN_PRIVILEGES (TRANSACT-SQL)

COLUMNS (Transact-SQL)

CONSTRAINT_COLUMN_USAGE (Transact-SQL)

CONSTRAINT_TABLE_USAGE (Transact-SQL)

DOMAIN_CONSTRAINTS (Transact-SQL)

DOMAINS (Transact-SQL)

KEY_COLUMN_USAGE (Transact-SQL)

PARAMETERS (Transact-SQL)

REFERENTIAL_CONSTRAINTS (Transact-SQL)

ROUTINE_COLUMNS (Transact-SQL)

ROUTINES (Transact-SQL)

SCHEMATA (Transact-SQL)

TABLE_CONSTRAINTS (Transact-SQL)

TABLE_PRIVILEGES

TABLES (Transact-SQL)

VIEW_COLUMN_USAGE (Transact-SQL)

VIEW_TABLE_USAGE

VIEWS (Transact-SQL)

See Also


Glossary (SQL Server PDW)

[image]

Glossary (SQL Server PDW)
See Also Send Feedback

This is the glossary of terms for SQL Server PDW.

Terms

Term

Definition

Admin Console

A web application that displays the appliance state information. Users can connect to the Admin Console through Windows Internet Explorer.

appliance

An application architected to run on specific hardware and software that function together as one box. Users can only access the box through the user interface of the application.

Appliance backup

An appliance backup is a backup of everything that is needed to restore the entire appliance. This includes a full (or full plus differential) backup of all user databases and a backup of the master database.

appliance-level collation

The collation that is applied to all metadata and databases in the SQL Server PDW appliance. When a database is created, the default collation is the appliance-level collation.

appliance node

A unit of hardware and software resources architected for a specific purpose. The SQL Server PDW architecture contains several types of appliance nodes.

Backup server

A server that is used for backup and restore operations. BACKUP DATABASE writes backups to the backup server. RESTORE DATABASE restores backups from the backup server.

This server is part of your own customer network and is not shipped with the appliance. SQL Server PDW uses a Windows file share to access data on the backup server. You can use the backup server for additional purposes, depending on your business requirements.

bi-level cache

A characteristic of the SQL Server PDW shared nothing architecture in which the Control node and each Compute node have separate query caches. This enables the each node to process queries locally occur without sharing the cache. The Control node query cache holds results for entire queries, whereas the Compute node query cache holds results for Compute node queries only.

collation

A collation specifies the rules for how strings of character data are sorted and compared, based on the norms of a particular language and locale. For example, in an ORDER BY clause, an English speaker would expect the character string 'Chiapas' to come before 'Colima' in ascending order. However, a Spanish speaker in Mexico might expect words beginning with 'Ch' to appear at the end of a list of words starting with 'C'. Collations dictate these kinds of sorting and comparison rules. The Latin_1 General collation will sort 'Chiapas' before 'Colima' in an ORDER BY ASC clause, whereas the Traditional_Spanish collation will sort 'Chiapas' after 'Colima'.

colocated data

Data that is stored and processed in the same location. In the SQL Server PDW appliance, data is colocated on each node. Colocation enables queries to run in parallel without impacting each other's performance.

complex join

A join on a non-distribution key column of a distributed table. This usually requires SQL Server PDW to copy data among the Compute nodes prior to performing the join.

Compute node

The basic unit of scalability for performance and storage. Each Compute node in the SQL Server PDW appliance uses its own user-data and computing resources to perform a portion of a parallel query. Because each Compute node is independent and self-sufficient, you can improve query performance on large data sets by adding more compute nodes.

The Compute nodes are the data storage and query processing backbone of the appliance. Each Compute node stores subsets of distributed data and performs massively parallel processing (MPP) query operations. Each Compute node receives query plan operations from the Control node, runs the query plan operations (which can be data transfer operations or Transact-SQL operations), and returns the results to the Control node.

Each Compute node in the SQL Server PDW appliance uses its own user-data and computing resources to perform a portion of a parallel query. Because each Compute node is independent and self-sufficient, you can scale out query performance and storage capacity by adding more Compute nodes.

Configuration Manager Tool

See SQL Server PDW Configuration Manager Tool.

Control node

Central point of control for processing queries on the SQL Server PDW appliance. The Control node receives the user query, creates a distributed query plan, communicates relevant plan operations and data to Compute nodes, receives compute node results, performs any necessary aggregation of results, and then returns the query results to the user. This node must be available in order to run queries on the appliance.

control node aggregation

Calculations performed on the Control node to aggregate the results from parallel queries. For example, the control node aggregates the results for the SUM and AVG function.

control rack

The control rack houses the servers, storage, and networking components for the nodes that provide control, management, or user interface functions. The control rack is the interface between the user and the Compute nodes.

control node saturation

A performance bottleneck on the Control node. A can occur when the Control node performs a large amount of aggregation.

control rack

Houses the hardware that provides control, management, and user interface functions for the SQL Server PDW appliance. The control rack is the interface between the user and the Compute nodes. There is only one control rack per appliance.

data distribution scheme

The strategy for storing tables across the SQL Server PDW appliance. For example, tables can be distributed (split, divided) or replicated (copied, duplicated).

data mart

A collection of read-only data used for a specific business purpose. The data mart is usually populated from a larger collection of data that is stored in a data warehouse.

Data Movement Service (DMS)

A SQL Server PDW service that manages communication and data transfer among the appliance nodes. This includes interfacing with the SQL Server instances in the SQL Server PDW appliance. DMS improves query performance by optimizing data transfer speeds.

Data provider

A layer of software that handles communication between data extensions and customized software specific to each type of external data source. Depending on the specific data source, multiple data providers are available from Microsoft and from third-party vendors.

data rack

The physical storage container for the compute node hardware. A data rack houses the servers, storage arrays, and networking components for a group of compute nodes. Each data rack within an appliance has the same number of Compute nodes; however, the number of Compute nodes in the data rack can vary among hardware vendors.

Because the data rack has the single purpose of housing Compute nodes, when designing the SQL Server PDW appliance, you can increase the number of Compute nodes by adding data racks.

data skew

A condition that occurs when the rows of a distributed table are not spread uniformly across all of the distributions. A significant variation in the number of rows in each distribution can negatively impact query performance.

To avoid data skew, choose a distribution key column whose values have a uniform distribution. For example, a column with random numbers will have a uniform spread across the distributions. Conversely, if a distribution column has repeating values such as time, date, or product price, the values can nucleate around a specific value and cause data skew.

data warehouse

A data warehouse is a collection of data that is loaded from one or more data sources and is used for performing business intelligence tasks such as reporting and data analysis. The data loading process, also called extract, transform and load (ETL), is a major component of the data warehouse.

differential backup

A database backup that only includes changes made since the last full backup. A differential backup can only be performed if the last full backup was completed successfully.

distributed relational database

A database that is controlled by a central database management system (DBMS) and stored as multiple collections of data that are not all attached to a common CPU. The SQL Server PDW appliance stores collections of data across multiple SQL Server database instances.

distributed table

A table whose rows are dispersed across multiple storage locations, called distributions.

distribution

The set of table rows, from one or more distributed tables, stored in the same location. SQL Server PDW uses the value in the distribution column to deterministically assign each row to a distribution.

To improve parallel query performance, the rows that will be joined together, in queries, should be stored together in the same distributions. This avoids transferring data among the distributions before running parallel query operations.

As part of the shared nothing SQL Server PDW architecture, each distribution is stored on its own set of disks. To accomplish this, the storage array for each Compute node is divided into disk groups (LUNs). Each distribution is stored on one LUN of one Compute node.

In SQL Server PDW, each Compute node stores eight distributions. For example, if there are eight Compute nodes, there will be 64 distributions (8 nodes x 8 distributions/node).

distribution column

The column whose values SQL Server PDW uses to assign a distributed table row to a distribution. Each distributed table has one column designated as the distribution column.

Domain Name System (DNS)

An internet standard for translating host names to IP addresses.

DNS Server

A software program that implements the DNS standard. Applications communicate with the DNS Server to translate host names into IP addresses

dwloader

A command-line tool which bulk loads rows into an existing table in the appliance. When loading rows, you can add all rows to the end of the table (append mode), append new rows and update existing rows (upsert mode), or delete all existing rows and insert all rows into an empty table (reload mode).

dwloader ships with the client tools. To use dwloader, install it onto your Loading server, and then run the tool on your Loading server.

explicit permission

A permission given on an object to a login or server role via the GRANT or DENY statements. This is in contrast to an implicit permission that is inherited from a server role.

full database backup

A backup of an entire SQL Server PDW database. A backup of a user database includes database users, and database roles. A backup of the master database includes logins.

InfiniBand

A high speed network used for private communication inside of a SQL Server PDW appliance.

implicit permission

A permission given on an object to a login or server role via inheriting an explicit GRANT or DENY permission from a server role. An object can only inherit an implicit permission if it does not have an explicit permission.

linear scale out

The SQL Server PDW process of adding more Compute nodes to gain a linear increase in query performance for a given dataset.

Loader

The APIs that SQL Server PDW uses to load data into the appliance. dwloader and the Integration Services Destination Adapters both load data by calling the Loader APIs.

Loading server

A server that stores data before it is loaded and provides the processing to copy data to the appliance.

This server is part of your own customer network and is not shipped with the appliance. You can use the Loading server for additional purposes, depending on your business requirements.

To load data, run dwloader command-line tool or Integration Services on the Loading server.

login

A password-protected user account for SQL Server PDW.

massively parallel processing (MPP)

The process of distributing the workload for a query across multiple CPUs or machines, resulting in faster performance.

MPP Engine

An appliance software component that runs on the Control node and coordinates these data warehousing functions that are specific to managing the SQL Server PDW application: Creates appliance-wide parallel query plans; coordinates parallel query execution on the Compute nodes; coordinates parallel transactions; stores and coordinates appliance-wide metadata and configuration data; manages appliance and database authentication and authorization; tracks hardware and software status.

MPP query plan

The "parallelized" query plan that the MPP engine creates to run a SQL user-query in parallel across the appliance nodes. The control query plan contains query operations organized into steps. The MPP Engine runs the steps sequentially. Each step can contain serial or parallel operations. All operations within a step must be completed before the MPP Engine runs the next step.

MPP query plan step

A step specifies one or more operations in the MPP query plan that must be run as a unit. For example, a step can contain multiple parallel SQL Server operations that must all complete on the Compute nodes before the engine can continue to the next step.

When specifying SQL Server operations, a step does not contain the SQL Server query plan. Instead, the step specifies the Transact-SQL statement; each SQL Server instance creates its own symmetric multiprocessing (SMP) query plan to run the Transact-SQL statement.

multi-level query cache

A characteristic of the SQL Server PDW shared nothing architecture in which each appliance node has its own query cache. This enables each node to process queries that locally occur without sharing the cache. For example, the Control node query cache holds the aggregated results for queries, whereas the Compute node query cache holds results for Compute node queries only.

parallel backup and restore

A feature of SQL Server PDW that performs parallel copies to backup a user database from the appliance nodes to the Backup server. Similarly, when data is restored in parallel from the Backup server to the appliance nodes.

remote table copy

A feature of SQL Server PDW that enables Hub and Spoke scenarios by copying the results of a SQL SELECT statement to a table in an SMP SQL Server database. The remote table copy is initiated with the CREATE REMOTE TABLE statement.

The SMP database is in an instance of SQL Server that is running on a Windows system which is attached to the internal InfiniBand network of the SQL Server PDW appliance. The SMP database cannot reside on a server within the SQL Server PDW appliance.

query parallelization

The process of splitting a SQL query into smaller "subqueries" that can be run simultaneously. Results from the subqueries are later recombined to produce a result set for the initial query.

redistribution

The process of copying data across the compute nodes in the appliance so that each distribution has all the distributed table data it needs to perform a complex join. Because a complex join is on one or more non-distribution columns, each distribution might not already have the correct data to perform its portion of the distributed complex join. Redistribution temporarily stores extra copies of row data so that each distribution can perform its portion of the complex join without retrieving data from another distribution.

replicated table

A table that is stored in its entirety on each compute node. Replicated tables require storage for each copy of the table, but do not need to be copied to additional Compute nodes before performing a join.

Server role

A group of principals (logins and server roles). Server roles are hierarchical. Logins and server roles can be added as members of one or more server roles. Use a server role to manage security permissions on securables (databases, tables, and views) for a group of logins and other membering server roles.

shared nothing (SN)

A distributed computing architecture in which each node contains its own data, CPU, and storage to function as a self-sufficient and independent unit.

simple join

A join on columns of replicated tables and/or distribution columns of distributed tables. There is no need to transfer data between distributions before performing a simple join.

In contrast, a complex join involves a non-distribution column of a distribution table. This requires copying data among the distributions before performing the join.

spare / standby

A hardware component that is kept available but not in use, intended to replace an existing appliance component in the case of a hardware failure.

There are three types of spares:

  • A hot spare is a hardware component that can be activated immediately and automatically by the appliance without administrator intervention.

  • A warm spare is a hardware component that requires minimal administrator intervention (such as entering commands or flipping switches) in order to become available to the appliance.

  • A cold spare is a hardware component that is not available to the appliance without significant administrator intervention and may require a significant amount of time to be brought online.

SQL Server

The database for managing data storage and query processing on the appliance. Each user database is distributed among multiple SQL Server database instances. Each Compute node runs an instance of SQL Server and uses it to manage user data. The Control node also runs an instance of SQL Server and uses it to manage metadata.

SQL Server PDW Configuration Manager

The SQL Server PDW Configuration Manager (PDWCM) is an appliance administration tool that SQL Server PDW system administrators use to perform appliance-level operations and to change appliance-level settings. For example, use PDWCM to reset passwords, set the time zone, change IP addresses, configure SSL certificates, enable remote access through the firewall, start or stop the appliance, and set Instant File Initialization.

Before using SQL Server PDW for the first time, use PDWCM to complete the setup process that could not be completed at the manufacturing facility due to security precautions and unknown user environment variables, such as IP addresses.

Staging database

A user-created SQL Server PDW database that stores data temporarily when it is being loaded into the appliance. When loading data, SQL Server PDW first copies the data to the staging database and then copies the data from temporary tables in the staging database to permanent tables in the destination database. If a staging database is not used, SQL Server PDW creates temporary tables in the destination database and uses them to store the loaded data before it inserts the loaded data into the permanent destination tables.

The primary benefit of a Staging database is to reduce table fragmentation. If a Staging database is not used, the data is loaded into temporary tables in the destination database. When temporary tables get created and dropped in the destination database, the pages for the temporary and permanent tables become interleaved. Over time, this causes table fragmentation and degrades performance. In contrast, a Staging database ensures that temporary tables are created and dropped in a separate file space than the permanent table.

super-user

A built-in account named the “SA” account. There is one super-user account per appliance. This account has all privileges.

Symmetric Multiprocessing (SMP)

A multi-processor computer architecture where identical processors share main memory. The architecture allows simultaneous processing to be distributed among the processors. In a symmetric multiprocessing (SMP) architecture, query processing occurs within one physical instance of a database. Conversely, in a massively parallel processing (MPP) architecture, query processing occurs within multiple instances of a database that each has dedicated CPU, memory, and storage.

See Also


HDInsight (Analytics Platform System)

[image]

HDInsight (Analytics Platform System)
See Also Send Feedback

HDInsight is one of the Analytics Platform System workloads. Use HDInsight on Analytics Platform System to create your own on-premises Hadoop solution, and to quickly integrate and analyze HDInsight data with SQL Server PDW data since all of the data is stored in close proximity on the same appliance.

HDInsight Benefits

Microsoft HDInsight is a 100 percent Apache Hadoop-based data platform that provides a Hadoop-based cloud solution for Windows Azure services, and an on-premises solution for Analytics Platform System. HDInsight manages data of any type, whether relational or nonrelational, and of any size. With APS, you can store and analyze HDInsight data on your own premises.

With APS, you can perform more complex analysis faster by using SQL Server PDW’s PolyBase to integrate analysis of your HDInsight non-relational data and your SQL Server PDW relational data, all on the same appliance.

HDInsight, running on APS, makes it easy to:

  • Load data of any structure into SQL Server PDW. By first loading your data into HDInsight, you can transform and process it, and then use PDW’s PolyBase to quickly load the data into SQL Server PDW relational tables.

  • Perform fast integrated analysis of HDInsight data and PDW data. The data are stored within close proximity of each other on the same appliance, making it fast to run integrated analysis with PDW’s PolyBase.

  • Export relational data to HDInsight region for a cold data store. To reduce the size of data stored in SQL Server PDW, you can move it to HDInsight and query the data at any time.

  • Trust that your data will be protected with the Analytics Platform System built-in high availability.

  • Monitor and manage HDInsight by using centralized appliance management tools.

For information, see HDInsight benefits on the Windows Azure web site.

In This Section

This section of the documentation contains the following sections.

  • Understanding the Security Model of the HDInsight Region (Analytics Platform System)

  • HDInsight Region Developer Dashboard (Analytics Platform System)

  • Using Hadoop Tools with APS (Analytics Platform System)

  • Loading Data into Hadoop (Analytics Platform System)

  • Tips for Using the REST API (Analytics Platform System)

  • Connecting to HDInsight Using Remote Desktop (Analytics Platform System)

  • Changing Hadoop Configuration Settings (Analytic Platform System)

  • Supported Hadoop Configuration Changes (Analytics Platform System)

  • Unsupported HDInsight Actions (Analytics Platform System)

See Also

Concepts

Appliance Management Tasks (Analytics Platform System)
Appliance Configuration (Analytics Platform System)


Understanding the Security Model of the HDInsight Region (Analytics Platform System)

[image]

Understanding the Security Model of the HDInsight Region (Analytics Platform System)
Send Feedback

The security structure of the HDInsight region of Microsoft Analytics Platform System is separate from the SQL Server PDW region of the Analytics Platform System. External access to the HDInsight region is controlled by the Secure Node Gateway which uses the internal Active Directory for credential and role membership checks. The Secure Gateway requires HTTP Basic Authentication. Credentials are included in the every request as part of HTTP authorization header. The appliance Active Directory is used to store user credentials, control user membership, and provide granular control of the features that are available for each security group (role).

The top level user of the HDInsight region is the Administrator user of the Analytics Platform System appliance domain (appliance_domain\Administrator). The Administrator user is created during the initial appliance deployment.

Within the HDInsight appliance, multiple cluster administrator accounts can be created as HDInsight users and multiple Information Worker accounts can be created as HDInsight users.

HDInsight Security Model Overview

The HDInsight region supports the authentication of multiple user accounts, but once authenticated, all Hadoop services and data access is made using the security context of system accounts provided by HDInsight. As a result, access to data and processing done by Map Reduce jobs, are not isolated from separate users. That is, jobs initiated by different users run under the same service account and have same privileges over the data.

HDP services run under Managed Service Accounts (MSA's) which are the optimal choice from a security point of view, in a distributed environment such as HDInsight cluster. MSA's enable automatic password management (passwords cannot be accessed or managed by human users), offer protection from interactive login, and simplified SPN management. For more information about MSA's, see Group Managed Service Accounts Overview.

Information Workers cannot remotely connect to the cluster (using Remote Desktop). Information Workers can access the system by using the Developer Dashboard, or by using the REST APIs that are exposed for various purposes. Those APIs can be invoked either directly (through common HTTP client tools such as wget, PostMan, HTTPie) or from applications that users develop on top of them.

There are three levels of privileges in HDI region which enables having different user profiles/roles optimized for subset of scenarios.

HDInsight region admin

A Windows login from the appliance Domain Administrators group. This is the single appliance_domain\Administrator account.

HDInsight cluster admin

A Power User, member of the HDInsight Cluster Admins group.

HDInsight regular user

A user account created by using the HDInsight Topology, User Management page of the DW Configuration Manager. These accounts are used by developers and information workers. Example: Melisa.

For information about creating users, see HDInsight User Management (Analytics Platform System).

Duties and Functions of the HDInsight Administrator Account

The top level user of the HDInsight region is the Administrator user of the appliance domain (appliance_domain\Administrator). The Administrator account is a member of the Domain Admins group of the appliance domain Active Directory. The Administrator user is created during the initial appliance deployment. The Administrator password can be changed using the DWConfig program, but the account cannot be renamed. Creating additional accounts which are member of Domain Admins group is not recommended.

The Administrator account has administrator level privileges on the HDInsight region organizational unit and all virtual machines.

In addition to having all the permissions of the HDInsight Cluster Admin, the Administrator account is used to impersonate access from Admin Console to HDI region. The Administrator account can be used also to:

  • Connect remotely to the physical host computers.

  • Connect remotely to the PDW region VM's (CTL01, etc.)

Duties and Functions of the HDInsight Cluster Administrator Accounts

The Cluster admin account uses a remote desktop connection to access cluster nodes to manage, configure, and troubleshoot Hadoop components (name node, resource manager, data node, node manager and others).

Connecting by using remote desktop is a two-step operation: Connecting to HDInsight Using Remote Desktop (Analytics Platform System)

The DW Configuration Manager can change the password for the domain admin account. For all other users, the password is changed on the Change password page of Developer Dashboard which is available during login.

In addition to having all the permissions of the HDInsight users, the HDInsight cluster administrators are members of the administrators group of the virtual machines, and can:

  • Connect by using remote desktop to the dedicated cluster nodes (virtual machines)

  • Collect and analyze HDP logs from cluster nodes to troubleshoot system issues

  • Use HDP tools to perform advanced administrative tasks which are not exposed directly by appliance tools. Examples include: Hadoop commands (fs, fsck, balancer, dfsadmin, jar, job), sqoop, and pig consoles.

  • Change whitelisted settings in Hadoop configuration files to adjust system to typical customer workloads. For more information, see Changing Hadoop Configuration Settings (Analytic Platform System).

  • Install and uninstall additional Hadoop modules (such as encryption and serialization libraries)

  • Create RunAs accounts to enable authentication of HDInsight or SCOM MP

Duties and Functions of the HDInsight User Accounts

Use the HDInsight User Management page of the DW Configuration Manager to create low privileged user accounts for information workers and developers.

User accounts can:

  • Access HDInsight services through the Gateway.

  • Use the Developer Dashboard to access Hive and to upload and download files.

  • Use the Change password page of the Developer Dashboard to change their own passwords. To access the Change password, on the Developer Dashboard log in page, click Change password.

For more information about creating users, see HDInsight User Management (Analytics Platform System).

Access to HDInsight Data

Hortonworks for Windows does not support user isolation therefore it’s not possible to run a job under security context of the user. In APS all HDInsight data access is performed by the Hadoop service account which has permission to read and alter any data in the HDInsight cluster. Therefore it is not possible to restrict an APS HDInsight user to only a limited subset of the data. One workaround for this problem is to access HDInsight data by using PolyBase queries that query HDInsight through a remote external table. You can use standard SQL Server permissions to control access to the remote external table.

Virtual Machines

The HDInsight Cluster Admins have administrator level privileges on HDInsight region virtual machines.

Remote desktop access by information workers to the new clusters is disabled by default. Enabling remote desktop access to non-admin users is not supported, necessary, or recommended.

Secure Perimeter

Access to HDInsight services is routed and secured through additional (non-HDP) components: Secure Node (HTTP gateway) and Developer Dashboard.

Secure node performs authentication and serves as a proxy for exposed set of Hadoop services:

  • WebHDFS for file manipulation (smaller files are targeted this way)

  • WebHCat (Templeton) for remote job submission and management

  • Oozie (for scheduling and executing complex workflows)

  • Hive2 server (for accessing HDInsight through the Hive ODBC driver)

The Developer Dashboard serves as an endpoint to Hive, job submission, and as a file upload/download utility through the WebHDFS. Access to Developer Dashboard requires the same credentials as access through Secure Node Gateway.

Secure Node Gateway communication only supports HTTPS.

Additional Information

For information about the system architecture, see the Microsoft Analytics Platform System section of Introduction (Analytics Platform System).

For information about SQL Server PDW authentication, see Security - Windows Authentication and SQL Server Authentication (SQL Server PDW).

For information about SQL Server PDW Active Directory trusts, see Security - Configure Domain Trusts (SQL Server PDW).


HDInsight Region Developer Dashboard (Analytics Platform System)

[image]

HDInsight Region Developer Dashboard (Analytics Platform System)
Send Feedback

Use the HDInsight Developer Dashboard to interact with the Hadoop components of the Analytics Platform System HDInsight Region. The Developer Dashboard is implemented as a web application that obtains information from the standard Hadoop components and tools. Use the Developer Dashboard to:

  • Quickly upload sample files and executables.

  • Quickly and easily run and monitor ad-hoc jobs.

  • Easily gain insight into Hive metadata, and to run and monitor queries.

  • Gain access to standard Hadoop web interfaces (Name Node UI and Resource Manager) for HDFS file browsing and download, name node and job tracker logs, etc.

To connect to the Developer Dashboard, using a web browser connect to the IP address of the HSN01computer, in the format https://xxx.xxx.xxx.xxx:81 When prompted, provide the credentials of your user account.

Security note Security Note

The Developer Dashboard does not pass credentials to other programs. Pages that navigate to Hadoop tools require you to login again. You can login with the same or with different credentials.

Main Page

APS HDI Developer Dashboard Top level

The Main page of the Developer Dashboard presents the following sections.

The Jobs area:

Title | Description
Create Job - HDInsight Region Developer Dashboard (Analytics Platform System) | Create Map/Reduce and Streaming jobs.
Hive Console - HDInsight Region Developer Dashboard (Analytics Platform System) | Provides the REPL experience to HDInsight developers, the ability to experiment with Hadoop data through the Hive QL interface.
Upload File - HDInsight Region Developer Dashboard (Analytics Platform System) | Copy a file to a selected path on the HDFS.
Job History - HDInsight Region Developer Dashboard (Analytics Platform System) | Uses the WebHCat tool to monitor the status of current and previous jobs.

The Status / Logs / Files area:

Title | Description
Name Node - HDInsight Region Developer Dashboard (Analytics Platform System) | Presents the Hadoop Namenode UI used for browsing the file system and accessing the NameNode log. Use the Name Node UI page to view cluster summary information (health).
Resource Manager - HDInsight Region Developer Dashboard (Analytics Platform System) | The default UI tool for inspection of job history on Hadoop. It displays all jobs, running, completed, failed and retired. This page gets data from the History Server and is not dependent on WebHCat. The Job History page provides a light weight view on the jobs submitted through WebHCat and also displays additional information provided by HDInsight.


Create Job - HDInsight Region Developer Dashboard (Analytics Platform System)

[image]

Create Job - HDInsight Region Developer Dashboard (Analytics Platform System)
See Also Send Feedback

Use the Create Job page of the HDInsight Developer Dashboard to submit jobs to HDInsight. The Create Job page presents two options; Standalone JAR or Hadoop Streaming. Select one to present that type of page.

Create Job

Standalone JAR

APS HDI Developer Dashboard Create Job

The Standalone JAR page has the following sections.

  • Job Name

    Provide a name for this new job.

  • Local JAR File

    Browse to select a file with the .JAR extension, containing MapReduce code.

  • Class Name

    Provide a class name.

  • Add parameter

    Click the Add parameter button to add a box where you can provide a parameter.

  • Final Command

    When parameters are added, displays the command as it is constructed.

  • Execute

    Creates the specified job.

To Submit a Job

  1. Open the Developer Dashboard. For more information on how to open the developer dashboard, see HDInsight Region Developer Dashboard (Analytics Platform System).

  2. Click on Create Job.

  3. In the Job Name box, type a name for this job.

  4. In the Local JAR File box, type a path and name, or browse to a job file.

  5. In the Class Name box, type a class name.

  6. If the job requires a parameter, click the Add parameter button and provide the parameter value.

  7. Click Execute to run the job.

Hadoop Streaming

HDI Region Developer Dashboard Create Hadoop Job

The Hadoop Streaming page has the following sections.

  • Job Name

    Provide a name for this new job.

  • Mapper

    Browse to select a mapper file.

  • Reducer

    Browse to select a reducer file.

  • Parameters

    Replace the content of the Parameter 0 edit box with -input <HDFS_path_to_input_file_or_directory> which represents input file(s) to the Mappers.

    Replace the content of the Parameter 1 edit box with -output < HDFS_path_to_output_directory> that will contain the result files that the Reducers output. This path must not exist in HDFS prior to creating this job.

See Also

Concepts

HDInsight Region Developer Dashboard (Analytics Platform System)


Hive Console - HDInsight Region Developer Dashboard (Analytics Platform System)

[image]

Hive Console - HDInsight Region Developer Dashboard (Analytics Platform System)
See Also Send Feedback

The Hive Console page provides the REPL experience to HDInsight developers; the ability to experiment with Hadoop data through the Hive QL interface. This console is accessed from the developer dashboard.

Hive Console

The console consists of two main components: The Object Explorer and a second component with both Editor and Queries tabs.

Object Explorer

The Object Explorer component represents available Hive objects in a Database/table/columns navigation tree. Double click on table objects in the Object Explorer to submit a query that retrieves the DDL statement of that table and automatically copy the DDL into the Query editor. After the query returns, the DDL can easily be modified and a similar table created. This feature is supported for databases and tables, but not for partitions.

Developer Dashboard Hive Object Explorer

Editor Tab

Use the Editor tab to create queries. When complete, click Run Query to submit the query. Optionally, provide a unique query title to help identify the query later in the Queries history section.

Developer Dashboard Hive Editor
Tip Tip

The Hive Console submits queries using WebHCat (Templeton). Follow WebHCat formats, such as enclosing literal values in single quotes not double quotes. For example, use 'WesternRegion' not "WesternRegion".

Queries Tab

The Queries tab contains a list of all the queries run in the current user session. Query execution details are listed: Query Title, Query Text, Status, Output, and Log. The Query Title and Query Text fields within a row, link back to the Editor window and populate the editor window with the query text. The Output and Log columns link to the query deep dive tab whereas query title, text, output and logs are shown.

Note Note

The Job History page keeps data across all user sessions in the dashboard because it is retrieved from the Job Tracker. The Queries tab is limited to the current user session.

HDI Region Developer Dashboard Hive Queries

See Also

Concepts

HDInsight Region Developer Dashboard (Analytics Platform System)


Upload File - HDInsight Region Developer Dashboard (Analytics Platform System)

[image]

Upload File - HDInsight Region Developer Dashboard (Analytics Platform System)
See Also Send Feedback

Use the Upload File page of the HDInsight Developer Dashboard to upload a JAR containing Hive UDFs that you want to use in Hive queries. To upload a file:

Upload File

APS HDI Developer Dashboard Upload File

  • Navigate to the File Upload page, click on the Upload tab, select Source (on the user’s local file system) and Destination (the absolute path in HDFS where the file will reside), and then to click Upload.

If the file already exists, a warning dialog box opens providing an opportunity to overwrite the file.

The file will be copied to the selected path in HDFS. After the operation is done, the status message is displayed to the user.

See Also

Concepts

HDInsight Region Developer Dashboard (Analytics Platform System)


Job History - HDInsight Region Developer Dashboard (Analytics Platform System)

[image]

Job History - HDInsight Region Developer Dashboard (Analytics Platform System)
See Also Send Feedback

Use the Job History page of the HDInsight Developer Dashboard to monitor the status of current and previous jobs. The Job History page uses the WebHCat tool to monitor the status of current and previous jobs. The Job History page displays the name, type, and arguments for each job, along with the start time and status. The information is taken directly from WedHCat job details.

Job History (WebHCat)

APS HDI Developer Dashboard Job History

The Map Reduce, Hive, and Map Reduce Streaming job types are available.

The Stop time for jobs is not available. Up to ten jobs are displayed by default. Click View All jobs to move to the Resource Manager view page.

Click a job name to display additional information about a job including the job output and execution log.

See Also

Concepts

HDInsight Region Developer Dashboard (Analytics Platform System)


Name Node - HDInsight Region Developer Dashboard (Analytics Platform System)

[image]

Name Node - HDInsight Region Developer Dashboard (Analytics Platform System)
See Also Send Feedback

Namenode allows you to browse the file system from the Developer Dashboard and get insight into name node logs. It is also possible to explore via WebHDFS, however the Name Node page is generally more convenient for manual browsing.

Name Node

APS Developer Dashboard NameNode

This page is part of Apache Hadoop and is exposed in the Developer Dashboard without any changes made by Microsoft.

See Also

Concepts

HDInsight Region Developer Dashboard (Analytics Platform System)


Resource Manager - HDInsight Region Developer Dashboard (Analytics Platform System)

[image]

Resource Manager - HDInsight Region Developer Dashboard (Analytics Platform System)
See Also Send Feedback

Use the Resource Manager page of the HDInsight Developer Dashboard to gain more insight and detailed information about job history, job tracker and task tracker logs.

Resource Manager

APS HDI Developer Dashboard Resource Manager
Note Note

This graphic represents only a portion of the available information.

When jobs are retired after they become invisible to WebHCat (and Job History) but they will still be visible from the Map Reduce UI page.

This page is part of Apache Hadoop and is exposed in the Developer Dashboard without any changes made by Microsoft.

See Also

Concepts

HDInsight Region Developer Dashboard (Analytics Platform System)


Using Hadoop Tools with APS (Analytics Platform System)

[image]

Using Hadoop Tools with APS (Analytics Platform System)
See Also Send Feedback

Standard Hadoop applications, tools, and techniques can be used with the Microsoft Analytics Platform System. In most cases, there are no behavior changes.

Supported Applications

Analytics Platform System includes an instance of Hortonworks Data Platform installation and supports the following Hadoop components.

Component | Version for APS AU1 | Version for APS AU2
Hadoop Core (HDFS, YARN, MapReduce) | 1.2.0 | 2.2.0
Hive | 0.11.0 | 0.12.0
HCatalog | 0.11.0 | 0.12.0
Pig | 0.11.0 | 0.12.0
Oozie | 3.3.2 | 4.0.0
Sqoop | 1.4.3 | 1.4.4

For more information about each component version, see the Hadoop documentation at Welcome to Apache™ Hadoop®!

Note Note

HeadNode is the virtual machine which hosts several HDInsight (Hadoop) master services: NameNode, Resource Manager, secondary NameNode, HiveMetaStore, HistoryServer, HiveServer2, OozieServer, and WebHcatServer (templeton).

Special Considerations

The following applications require special techniques.

Hive

If you are using the Microsoft Hive ODBC Driver to connect to HDInsight from BI, analytics, and reporting tools, a trusted certificate is required.

Oozie

To submit Oozie jobs use the command line tools installed on the Head node or submit a workflow remotely through Oozie REST API.

When creating Oozie jobs, the action configuration block of the workflow.xml file, must specify a separate queue named joblauncher, for the Oozie job launcher. The joblauncher queue has the capacity of 25% of the total cluster capacity. This queue is only for map only control tasks. To direct Oozie controller tasks to this queue, add oozie.launcher.mapred.job.queue.name=joblauncher to the action configuration block in the workflow.xml file.

For more information about Oozie, see Chapter 4. Using HDP for Workflow and Scheduling (Oozie).

Using the Hadoop REST API's

Developers can submit requests by using the standard Hadoop REST APIs such as WebHDFS, WebHCat, and the Oozie Web Services API. Requests should be directed to the IP address of the HSN node (gateway). A TCP port number is not necessary. Provide basic authentication credentials (username, password) in the request header.

Additional References

For more information about REST APIs, see WebHDFS REST API, Chapter 1. Using WebHDFS REST API and 2. Using WebHCat

See Also

Concepts

Loading Data into Hadoop (Analytics Platform System)


Loading Data into Hadoop (Analytics Platform System)

[image]

Loading Data into Hadoop (Analytics Platform System)
See Also Send Feedback

When loading data into Hadoop, the Analytics Platform System HDInsight Region supports standard Hadoop methods such as using a map-only job for batch load, using the WebHDFS REST API for trickle-load, and using Sqoop. You can also load data using Polybase, and the HDInsight Developer Dashboard.

Note Note

HDInsight running on Windows Azure stores data using Windows Azure blob storage (WASB). The HDInsight Region of Analytics Platform System stores data by using HDFS.

HDInsight data loading scenarios

Different scenarios call for different methods of loading data into the HDInsight cluster. In this section five scenarios are discussed:

  • Using a Hadoop map job for batch data load

  • Using trickle load via the WebHDFS REST API

  • Using Sqoop for loading data from external relational sources

  • Using PolyBase, for archiving relational data

  • Using Developer Dashboard to upload files

Note Note

The examples in this topic present IP addresses with the first two octets masked as xxx.xxx and passwords masked as **********. Modify the examples with data that matches your environment.

Using a Hadoop map job for batch data load

Use a map-only Hadoop job when loading a large amount (a batch) of data from the corporate network. The data source can be any location accessible to the customer but most often this is a network share with multiple files containing data to be loaded at about the same time. A typical example is a job regularly scheduled to run at a predefined time (usually once a day), that will load, curate, and analyze all incremental data changes that occurred since the previous job run.

Network configuration

The network share that contains the data for loading should be configured to be accessible from the HDI region HeadNode. How to enable this depends on network settings. Typical items to configure include making sure that file sharing is allowed, the firewall is not preventing file sharing (check outbound data access for HDInsight in DW Config), etc.

To protect data transferred over the network, administrators should configure SMP 3.0 or to consider other mechanisms to ensure data privacy on their shares.

After configuring a shared folder, the administrator should double-check that the settings are correct trying to access the data source by using windows explorer:

Access a network share using Windows Explorer

Before getting access the user will be prompted for credentials. It is good practice to use credentials with low (read-only) privileges and limited validity period, as the credentials may be visible in the Hadoop logs if provided as job input parameters. Once the administrator makes sure the data is accessible and available, a map job can be configured for data loading.

Configuring and invoking a data loading job

The following sections describe a typical example of a data loading job. The method explained here (and the code provided) is not the only solution to this problem but it illustrates one possible approach.

The jar file with the code must be uploaded into HDFS.

Important note Important

When you build the .jar file make sure to use Java for Windows that matches the version deployed on your APS appliance.

All input files describing the load should be placed in HDFS. The following is the input file format for the provided code example:

Copy image Copy Code
SomeSharedFolder#SourceFileName1#HDFSDestinationPath#DestinationFilename1
SomeSharedFolder#SourceFileName2#HDFSDestinationPath#DestinationFilename2
…
SomeSharedFolderY#SourceFileNameX#HDFSDestinationPathZ#DestinationFilenameX

Character # is used as delimiter which enables the map job to distinguish the following data elements:

  • Name of the folder at source

  • File name at source

  • Destination folder in HDFS

  • File name in HDFS

Here is an example for the file content. Replace the IP address at the start of each line with an appropriate IP address in your environment:

Copy image Copy Code
\\xxx.xxx.255.79\CorpShare\FileShare\#FileFromShare001.txt#/sp/lfs/t001/data/#FileFromShare001.txt
\\xxx.xxx.255.79\CorpShare\FileShare\#FileFromShare002.txt#/sp/lfs/t001/data/#FileFromShare002.txt
\\xxx.xxx.255.79\CorpShare\FileShare\#FileShare01.txt#/sp/lfs/t001/data/#FileShare01.txt
\\xxx.xxx.255.79\CorpShare\FileShare\#FileShare02.txt#/sp/lfs/t001/data/#FileShare02.txt
\\xxx.xxx.255.79\CorpShare\FileShare\#FileShare03.txt#/sp/lfs/t001/data/#FileShare03.txt
\\xxx.xxx.255.79\CorpShare\FileShare\#FileShare04.txt#/sp/lfs/t001/data/#FileShare04.txt
\\xxx.xxx.255.53\Share3\Files\#file1.txt#/sp/lfs/t001/data/#File1.txt
\\xxx.xxx.255.53\Share3\Files\#file2.txt#/sp/lfs/t001/data/#File2.txt
\\xxx.xxx.255.53\Share3\Files\#file3.txt#/sp/lfs/t001/data/#File3.txt
\\xxx.xxx.255.53\Share3\Files\#file4.txt#/sp/lfs/t001/data/#File4.txt

The next portion of this example shows how to invoke the load code. This example assumes that jar with map job is located in the HDFS path /CustomCode/Load/LoadFromShare.jar, and the input file is located at /Data/Load/Load1/InputFile1.txt. Note that the job expects named parameters for input_path, output_path, and network_shares. If you don't specify the parameter names correctly the job will end with an error message.

You can invoke data loading job using one of the following options:

  • Run curl.exe externally and invoke the Templeton (WebHCat) REST API.

    Example command line for job invocation using curl and WebHCat:

    Copy image Copy Code
    curl.exe "https://xxx.xxx.217.107/templeton/v1/mapreduce/jar" -s -d
user.name=Administrator -d jar="/CustomCode/Load/LoadFromShare.jar" -d
class=loadfromshare.LoadFromShare -d arg=input_path:/Data/Load/Load1/ -d
arg=output_path:/Data/Load/Load1Result/ -d arg="network_shares:
\\xxx.xxx.255.79\CorpShare\FileShare,Password1,LocalTestUser#
\\xxx.xxx.217.98\Share3\Files,Password2,CORPDOMAIN\DomainAdmin" -i -u
Administrator:********* –k

  • Run the hadoop jar command from the HDInsight head node.

    Example command line for job invocation hadoop jar (assuming that loadfromshare.jar is located in the c:\LoadFromShare folder):

    Copy image Copy Code
    c:\LoadFromShare>hadoop jar loadfromshare.jar
loadfromshare.LoadFromShare "input_path:/Data/Load/Load1/"
"output_path:/Data/Load/Load1Result/" "network_shares:
\\xxx.xxx.255.79\CorpShare\FileShare,Password1,LocalTestUser#
\\xxx.xxx.217.98\Share3\Files,Password2,CORPDOMAIN\DomainAdmin"
Note Note

The curl tool can be downloaded from cURL Releases and Downloads.

The following table describes parameters used in the command above:

Parameter | Value Provided | Comment
Secure Node Gateway | xxx.xxx.217.107 (Replace with your IP address) | Address of the Secure Node is necessary when job is submitted through WebHCat (Templeton) API.
User.name | Administrator | User which submits request. It should align with credentials provided in –u parameter.
"input_path" | /Data/Load/Load1/ | HDFS path that will be used as the map-reduce input. This path may contain one or more input files. In general there may also be many input paths but code provided here assumes there is only one.
"output_path" | /Data/Load/Load1Result/ | HDFS path that will be used as the map-reduce output. This path must not exist when job is started. If it exists, the job will fail.
"network_shares" | LocalTestUser#\\xxx.xxx.217.98\Share3\Files,Password2,CORPDOMAIN\DomainAdmin | Argument containing information on credential to access file shares: file share path, username and password.
u | Administrator:********* (Replace with your administrator password.) | HTTP basic authentication credentials of the user who submits the WebHCat (Templeton) request.

Data loading - code example

The code that performs the load is a map-only job. That means that mappers will copy the data, and that reducer does not exists because there is nothing that should be done after copying. Reducer might be a good place for some custom code that creates a data load summary (total amount of loaded data, how much time data load took, etc.), but this is not covered by this example.

This code example has two classes: LoadFromShare and LoadFromShareMapper which extends Mapper class.

LoadFromShare

In LoadFromShare class we setup the map-reduce job. There are a few specific for this setup. We use NLineInputFormat. This means that each mapper will process only N lines from the input file (there may be many input files). In the code below N equals 1. The rationale behind this, is that file copy is long operation and the overhead of starting map task for each file copy is small. If that is not true (when data loading deals with large number of small files), then the developer should consider increasing N from 1 to some greater value. Additionally, a third parameter is processed and values from it are added to job configuration. By doing this we enable each mapper to have information on shares and credentials for them.

LoadFromShareMapper

LoadFromShareMapper code in setup reads info on shares from the configuration and calls NET USE command to get rights to access the shares. Mapper as input (value parameter) gets one line of an input file, extracts from it source and destination for copy operation and then copies the file in chunks of the size of the buffer. When certain amount of data is copied mapper sends progress signal to the job tracker. This is done because copy might take a lot of time and long-running unresponsive tasks are killed by the JobTracker. By calling the progress method, mapper tells JobTracker that it is alive.

Note Note

The code sample below is an example how job for data loading might look like. It’s is not designed to catch and surface every possible error that might happen in a real use case. You should adjust it to your specific environment. Keep in mind following issues:

  • If your .jar file is corrupted (or compiled with the wrong JDK version), the error message returned in the case of WebHDFS submission won’t properly indicate root cause of the problem.

  • The NET USE command generated from java code may fail silently for some reason (there is a comment on this in the code). You should check whether a manually submitted NET USE command works properly in your environment.

Copy image Copy Code
package loadfromshare;

import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.lib.input.NLineInputFormat;

/**
*
* @author
*/
public class LoadFromShare {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws Exception {

        String inputPathParam = null;
        String outputPathParam = null;
        String networkSharesParam = null;

        String inputPathParamPrefix = "input_path:";
        String outputPathParamPrefix = "output_path:";
        String networkSharesParamPrefix = "network_shares:";

        for(int i = 0; i < args.length; i++)
        {
            System.out.println("args[" + i + "] " + args[i]);
            if (args[i].indexOf(inputPathParamPrefix) == 0)
            {
                inputPathParam = args[i].substring(inputPathParamPrefix.length(), args[i].length());
            }
            else if (args[i].indexOf(outputPathParamPrefix) == 0)
            {
                outputPathParam = args[i].substring(outputPathParamPrefix.length(), args[i].length());
            }
            else if (args[i].indexOf(networkSharesParamPrefix) == 0)
            {
                networkSharesParam = args[i].substring(networkSharesParamPrefix.length(), args[i].length());
            }
        }

        if (inputPathParam == null || outputPathParam == null || networkSharesParam == null)
        {
            throw new Exception("Input parameters are invalid. You must specify 'input_path:', 'output_path:' and  'network_shares:' .");
        }
        System.out.println("inputPathParam: " + inputPathParam);
        System.out.println("outputPathParam: " + outputPathParam);
        System.out.println("networkSharesParam: " + networkSharesParam);

        Job job = new Job();
        job.setJarByClass(LoadFromShare.class);
        job.setJobName("LoadFromShare");
       job.setMapperClass(LoadFromShareMapper.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);

        // When NLineInputFormat is used, a map task is started
        // for each N lines in the input file.
        // Without this, map task would be started
        // for each file in the input.
        //
        job.setInputFormatClass(NLineInputFormat.class);

        // If files that should be copied are large, this setting (value == 1) is good.
        // If files are small, consider increasing number of lines per split.
        // This would decrease overhead of starting map task.
        //
        NLineInputFormat.setNumLinesPerSplit(job, 1);

        FileInputFormat.addInputPath(job, new Path(inputPathParam));
        FileOutputFormat.setOutputPath(job, new Path(outputPathParam));

        String [] networkSharesInfo = networkSharesParam.split("#");
        Integer numberOfShares = networkSharesInfo.length;
        for(int i = 0; i < networkSharesInfo.length; i++)
        {
            String info = networkSharesInfo[i];
            System.out.println("--Share_" + i + " ; " + info);
            job.getConfiguration().set("Share_" + i, info);
        }
        System.out.println("--NumberOfShares" + " ; " + numberOfShares.toString());
        job.getConfiguration().set("NumberOfShares", numberOfShares.toString());
        job.getConfiguration().setInt("mapred.reduce.tasks", 0);

        System.exit(job.waitForCompletion(true) ? 0 : 1);
    }
}

package loadfromshare;

import java.io.IOException;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;
import java.io.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.conf.*;

/**
*
* @author
*/
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;

public class LoadFromShareMapper extends Mapper<LongWritable, Text, Text, IntWritable> {

    public int ioBufferSizeMB = 16;

    @Override
    public void setup(Context context)
            throws IOException
    {
        int numberOfShares = Integer.parseInt(context.getConfiguration().get("NumberOfShares"));
        System.out.println(">NumberOfShares: " + numberOfShares);
        System.out.println(">Share_0: " + context.getConfiguration().get("Share_0"));
        for(Integer i = 0; i < numberOfShares; i++)
        {
            String key = "Share_" + i.toString();
            System.out.println(">key: " + key);
            String share = context.getConfiguration().get(key);
            System.out.println(">share: " + share);
            String[] tokens = share.split(",");
            String netuseCommand = "cmd /c net use " + tokens[0] + " " + tokens[1] + " /USER:" + tokens[2] + " /persistent:yes";
            System.out.println(">cmd: " + netuseCommand);
            Process process = Runtime.getRuntime().exec(netuseCommand);
            try
            {
                // Note that output of NET USE is not verified in this example.
                // If for any reason this command fails, that will be silent at this point,
                // but may cause failure in later file copy operations.
                //
                process.waitFor();
            }
            catch (InterruptedException e)
            {
                System.out.println("InterruptedException occured");
                System.out.println(e.toString());
            }
            OutputStream os = process.getOutputStream();
            PrintStream ps = new PrintStream(os);
            ps.println();
        }
    }

    @Override
    public void map(LongWritable key, Text value, Context context)
            throws IOException, InterruptedException {

        String[] params = value.toString().split("#");
        String sharedFolder = params[0];
        String sharedFile = params[1];
        String hdfsPath = params[2];
        String hdfsFile = params[3];

        FileSystem hdfs = FileSystem.get(new Configuration());
        System.out.println("Creating outFile: " + hdfsPath + hdfsFile);
        FSDataOutputStream outFile = hdfs.create(new Path(hdfsPath + hdfsFile));
        System.out.println("Creating inFile: " + sharedFolder + sharedFile);

        try {
            FileInputStream in = new FileInputStream(sharedFolder + sharedFile);

            System.out.println("Done creting files.");

            long progressSize = 64 * 1024 * 1024l;
            long copiedSize = 0;
            byte[] buf = new byte[ioBufferSizeMB * 1024 * 1024];
            int len = 0;

            while ((len = in.read(buf)) > 0) {
                outFile.write(buf, 0, len);
                copiedSize += len;

                // Copying data may be a long running operation.
                // Each time size of copied data reaches a certain
                // (relatively small) limit mapper sends progress signal.
                // This prevents job tracker to kill long running mapper.
                //
                if (copiedSize >= progressSize) {
                    copiedSize = 0;

                    // Tell job tracker that this mapper is alive and
                    // should not be killed, even though it is long running.
                    //
                    context.progress();
                }
            }

            in.close();
            outFile.close();
        }
        catch (Exception e)
        {
            System.out.println(e.toString());
        }
    }
}

Compiling the code

Follow the next steps to correctly compile java code:

  1. Create a folder c:\loadfromshare on the HDInsight HeadNode and copy the two .java files (LoadFromShareMapper.java and LoadFromShare.java) to that folder.

  2. Run two commands to compile .java files in the following order:

    Copy image Copy Code
    c:\LoadFromShare>C:\Java\jdk1.7.0_51\bin\javac -classpath
C:\hadoop\hadoop-2.2.0.2.0.8.0-1639\share\hadoop\mapreduce\
hadoop-mapreduce-client-core-2.2.0.2.0.8.0-1639.jar;
C:\hadoop\hadoop-2.2.0.2.0.8.0-1639\share\hadoop\common\
hadoop-common-2.2.0.2.0.8.0-1639.jar -d C:\loadfromshare\
C:\loadfromshare\LoadFromShareMapper.java
    Copy image Copy Code
    c:\LoadFromShare>C:\Java\jdk1.7.0_51\bin\javac -classpath
C:\hadoop\hadoop-2.2.0.2.0.8.0-1639\share\hadoop\mapreduce\
hadoop-mapreduce-client-core-2.2.0.2.0.8.0-1639.jar;
C:\hadoop\hadoop-2.2.0.2.0.8.0-1639\share\hadoop\common\
hadoop-common-2.2.0.2.0.8.0-1639.jar;;c:\loadfromshare
-d C:\loadfromshare\ C:\loadfromshare\LoadFromShare.java

  3. Use the jar command to package the class files from the root folder (in this case: c:\loadfromshare):

    Copy image Copy Code
    c:\LoadFromShare>C:\Java\jdk1.7.0_51\bin\jar cvf
loadfromshare.jar .\loadfromshare\*.class

(top)

Using trickle load via the WebHDFS REST API

This method of loading is convenient to store data that is generated by multiple application instances (data sources) where input files are usually relatively small (compared to batch load) but loading frequency is considerably high and data loading is performed in a concurrent, parallel fashion. Typical examples include data continuously fed by sensors, power meters, or applications that continuously append incremental changes to their log files. In this scenario, the best approach is to load information by using an application that uses the WebHDFS API.

The following example illustrates how this API is used from one common HTTP tool – curl. Loading data with WebHDFS is two-step operation. In the first step, the client application specifies file path (including the file name) where data will be stored. Similar to WebHCat, the user.name specifies identity of the user and the –u parameter is used for HTTP basic authentication:

Copy image Copy Code
curl -i -X PUT "https://xxx.xxx.217.107/webhdfs/v1/trickleload/file3.txt?user.name=HDIUser&op=CREATE" -k -u HDIUser:Password1 -H Content-Length:0
Received response is a temporary redirect:
HTTP/1.1 307 TEMPORARY_REDIRECT
Content-Length: 0
Content-Type: application/octet-stream
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Location: https://xxx.xxx.217.107/rwebhdfs/H16021-C-HDN002/50075/webhdfs/v1/trickleload/file3.txt?op=CREATE&user.name=Administrator&overwrite=false
Server: Microsoft-IIS/8.0
Set-Cookie: hadoop.auth="u=Administrator&p=Administrator&t=simple&e=1386738798008&s=xJtW0Fc8s484kofGte2oQuH/iJs=";Path=/
X-Powered-By: ARR/2.5
X-Powered-By: ASP.NET
Date: Tue, 10 Dec 2013 19:13:18 GMT

The received location should be used for next request. Along with the redirect location, the second request must contain a local file path, user.name, and credentials for HTTP basic authentication:

Copy image Copy Code
curl -i -X PUT -T c:\tmp\file3.txt "https://xxx.xxx.217.107/
rwebhdfs/H16021-C-HDN002/50075/webhdfs/v1/trickleload/file3.txt?op=
CREATE&user.name=Administrator&overwrite=false" -k
-u Administrator:*********

The final response contains the status of the entire operation:

Copy image Copy Code
HTTP/1.1 201 Created
Content-Length: 0
Content-Type: application/octet-stream
Location: webhdfs://xxx.xxx.217.107:443/trickleload/file3.txt
Server: Microsoft-IIS/8.0
X-Powered-By: ARR/2.5
X-Powered-By: ASP.NET
Date: Tue, 10 Dec 2013 19:14:28 GMT

WebHDFS can be used for other file manipulation, including reading existing files, folder creation and renaming, file deletion, etc. The full WebHDFS reference can be found at WebHDFS REST API

(top)

Using Sqoop for loading data from external relational sources

Use Sqoop to transfer data from relational data sources into the HDInsight for analysis. (Use PolyBase when the source of the data is SQL Server PDW.) To execute the Sqoop command, the administrator usually needs to use Sqoop command line tool located on the headnode. This requires a remote connection to the cluster. Often, Sqoop is part of complex data loading workflow, where it can be one of several workflow actions. In that scenario, another REST API (Oozie) is used to define and run the data loading workflow. Since Oozie is exposed externally (similarly to other REST APIs), this approach doesn’t require a remote connection to the head node which means that regular HDInsight users (without cluster admin permissions) can invoke Oozie and Sqoop.

The following example illustrates using single theSqoop command from the command line tool. First the user connects remotely to the headnode. Then the user opens the command line tool and goes to directory C:\hadoop\sqoop-1.4.3.1.3.0.0-0539. (The HDP version number may be different on your appliance.)

The following example loads data using Sqoop from the command line:

Copy image Copy Code
sqoop import --driver com.microsoft.sqlserver.jdbc.SQLServerDriver
--connect "jdbc:sqlserver://xxx.xxx.30.40:1433;Database=DatabaseName"
--table TableName --username User001 -P --fields-terminated-by \t –m 1
 –target-dir /user/MyUserName/DatabaseName/TableName/

In this example, xxx.xxx.30.40:1433 is the IP address and TCP port number of the instance of SQL Server that contains the data.

In order to use Sqoop, outbound connections must be enabled by a firewall rule in DW Config. HDInsight only supports SQL Server Authentication for Sqoop commands.

The example loads the entire TableName table from DatabaseName database into the HDFS path /user/MyUserName/DatabaseName/TableName/. Sqoop has many more options and variations of how data load can be done.

(top)

Using PolyBase, for archiving relational data

Use PolyBase to move older or less relevant data out of SQL Server PDW into HDInsight to reduce the price of storing data. This cold data store scenario archives older data in a way that the data still can be queried in a truly unified way along with the current relational data that is stored in PDW.

The mechanism which enables this scenario (archiving and unified querying) is PolyBase, a specific PDW feature. It enables users to create external tables in HDInsight and then use those tables as any other relational table (and in conjunction with relational table). Although external tables can be created on existing HDFS files, in this scenario we are particularly interested in external tables that are created and loaded based on data that is stored in PDW tables.

The LOCATION must specify the HeadNode for your HDInsight workload. The following example loads data from the AdventureWorks2012PDW.dbo.dimCustomer table into HDFS on the path /database/HDI_dimCustomer. <HDInsight Domain Name>-C-HHN01 is the name of the HeadNode on the internal InfiniBand network.

Copy image Copy Code
CREATE EXTERNAL DATA SOURCE HDI_source
WITH (
    TYPE = HADOOP,
    LOCATION = hdfs://<HDInsight Domain Name>-C-HHN01:8020'
)

CREATE EXTERNAL FILE FORMAT HDI_format
WITH (
    FORMAT_TYPE = DELIMITEDTEXT,
    FORMAT_OPTIONS (FIELD_TERMINATOR ='|')
);

CREATE EXTERNAL TABLE HDI_Table
WITH (
        LOCATION='/ database/HDI_Table',
        DATA_SOURCE = HDI_source,
        FILE_FORMAT = HDI_format
    )
AS SELECT * FROM AdventureWorks2012PDW.dbo.dimCustomer
;

This command also creates an external table in PDW with the name HDI_dimCustomer. After execution of this command, the HDI_dimCustomer file in HDFS will be populated with data from AdventureWorks2012PDW.dbo.dimCustomer. Once the external table is created it can be used in PDW DSQL queries as if it was created directly upon HDFS data.

PolyBase queries always access the HDInsight region directly (bypassing the Secure Node Gateway) by specifying the IP addresses or names from the internal InfiniBand network. No authentication is required for this type of request, regardless if data is transferred from HDInsight to PDW or the other way around (as in this example). For more information, see PolyBase for Hadoop and WASB Integration (SQL Server PDW).

(top)

Using Developer Dashboard to upload files

Use the HDInsight Developer Dashboard for loading smaller utility files that support execution of MapReduce jobs or Oozie workflows, submitted and executed through REST API channels. Data loading in this scenario is limited in several ways: it’s manual, covering only files from client local file system, and supporting a size equal or less to 20 MB. However, this scenario, has its place as a file upload utility that facilitates job submission scenarios. For more information about uploading files with Developer Dashboard, see Upload File - HDInsight Region Developer Dashboard (Analytics Platform System).

(top)

See Also

Concepts

Using Hadoop Tools with APS (Analytics Platform System)
Tips for Using the REST API (Analytics Platform System)


Tips for Using the REST API (Analytics Platform System)

[image]

Tips for Using the REST API (Analytics Platform System)
See Also Send Feedback

The HDInsight region of Analytics Platform System supports access from tools and applications using the Hadoop REST API's.

In This Topic

  • All API's

  • Remotely accessing HDFS (WebHDFS)

  • Remote Job Submission (WebHCat)

  • Remote Workflow Submission and Scheduling (Oozie)

All API's

Important notes for all APIs:

  • Only HTTPS is enabled

  • No port number is required in the URI

  • Specification of user identity in the request is mandatory (anonymous requests are not allowed)

    • The Secure Node Gateway performs authentication on behalf of the end points

    • The end points themselves are not performing additional authentication but are accepting the user identity as a tag for requests and transferring the identity correctly into the Hadoop cluster (i.e. ownership of files/folders, jobs, etc.).

    • For additional information about the HDInsight security model, see Understanding the Security Model of the HDInsight Region (Analytics Platform System).

  • PUT requests must contain a specified content length. A length of 0 can be provided. This is a requirement imposed by the Secure Node Gateway.

The following examples for accessing Rest API use curl command line tool, a commonly used REST API client. Any REST API client supporting HTTP basic authentication can be used. In addition, users can submit requests from custom applications (.NET or other).

Note Note

The curl tool can be downloaded from cURL Releases and Downloads.

Remotely accessing HDFS (WebHDFS)

HDInsight exposes the WebHDFS REST API end point through the Secure Node Gateway. The API is used for HDFS file folder browsing and manipulation. The Gateway acts as a proxy for all requests sent to the cluster.

Copy image Copy Code
--List status of the user’s home directory (replace <username> and <password> with your account data)
curl -i -k -u <username>:<password> "https://<secure_node_IP>/webhdfs/v1/user/<username>?user.name=<username>&op=LISTSTATUS"

--Create file in user’s home directory: request 1
curl -i -H "Content-Length:0" -k -u <username>:<password> -X PUT "https://<secure_node_IP>/webhdfs/v1/user/<username>/hellowebhdfs.txt?user.name=<username>&op=CREATE"

--Response (part)
HTTP/1.1 307 TEMPORARY_REDIRECT
Content-Length: 0
Content-Type: application/octet-stream
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Location: https:// <secure_node_IP>/rwebhdfs/<datanode_name>/50075/webhdfs/v1/user/<username>/hellowebhdfs.txt?op=CREATE&user.name=<username>&overwrite=false

--Upload file content (redirect): request 2
--(can be automatically handled by curl if –L option was used in initial request)
curl -i -k -u <username>:<password> -X PUT -T "C:\tests\hellowebhdfs.txt" "https:// <secure_node_IP>/rwebhdfs/<datanode_name>/50075/webhdfs/v1/user/<username>/hellowebhdfs.txt?op=CREATE&user.name=<username>&overwrite=false"

Remote Job Submission (WebHCat also known as Templeton)

HDInsight exposes the WebHCat REST API end point through the Secure Node Gateway. The API is used for job submission, monitoring, and management. The Gateway acts as a proxy for all requests sent to the cluster.

Copy image Copy Code
--Check the status of the WebHCat service
curl -k -u <username>:<password> "https://<secure_node_IP>/templeton/v1/status?user.name=<username>"

--Response (if all OK)
{"status":"ok","version":"v1"}

--Submit new job (Hive query)
curl -i -k -u <username>:<password> -d execute="show tables;" –d statusdir="/user/<username>/hive/show_tables_output" "https://<secure_node_IP>/templeton/v1/hive?user.name=<username>"

--Response (part)
{"id":"job_201312102153_0011"}

--Get job status (for a given job_id)
curl -k -u <username>:<password> "https://<secure_node_IP>/templeton/v1/jobs/job_201312102153_0011?user.name=<username>"

--Cancel executing job
curl -X DELETE  -k -u <username>:<password> "https://<secure_node_IP>/templeton/v1/jobs/ job_201312102153_0011? user.name=<username>"

Remote Workflow Submission and Scheduling (Oozie)

HDInsight exposes the Oozie REST API end point through the Secure Node Gateway. The API is used for workflow and job submission, scheduling and monitoring, and management. Workflows usually consist of multiple jobs and serve the purpose of end-to-end operationalization. The Secure Node Gateway acts as a proxy for all requests sent to the cluster.

Specific notes for this API: Specification of the user identity is not done as part of URL but rather through the user.name property in the XML content of the request. This is demonstrated in the POST example below.

Copy image Copy Code
--Check the status of the Oozie service / get version
curl –i -k -u <username>:<password> "https://<secure_node_IP>/oozie/versions "

--Response (if all OK)
HTTP/1.1 200 OK
Content-Length: 5
Content-Type: application/json;charset=UTF-8
…
[0,1]

--Submit new workflow
curl -k -X POST -H "Content-Type: application/xml" -u <username>:<password> "https://<secure_node_IP>/oozie/v1/jobs" -d @<local_path>/oozie-example.xml
where oozie-example.xml is something like:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property>
<name>user.name</name>
<value><username></value>
</property>
<property>
<name>oozie.wf.application.path</name>
<value>hdfs:///<hdfs-path-to-workflow-definition></value>
</property>
</configuration>

--Response (part)
{"id":"0000000-131210215336175-oozie-Hado-W"}

--Start the workflow execution
curl -i -k -X PUT -H "Content-Length:0" -u <username>:<password> "https://<secure_node_IP>/oozie/v1/job/0000000-131210215336175-oozie-Hado-W?action=start"

--Inquire the job status
curl -i -k -u <username>:<password> "https://<secure_node_IP>/oozie/v1/job/0000000-131210215336175-oozie-Hado-W?show=info"

See Also

Concepts

Loading Data into Hadoop (Analytics Platform System)
Using Hadoop Tools with APS (Analytics Platform System)


Connecting to HDInsight Using Remote Desktop (Analytics Platform System)

[image]

Connecting to HDInsight Using Remote Desktop (Analytics Platform System)
See Also Send Feedback

For perform advanced cluster administration, remotely connect to virtual machines in the HDI region.

Note Note

Connecting to the physical machines (from the appliance domain) is not covered by this instruction.

Scenarios

Connect to the virtual machines for advanced cluster administration.

  • Troubleshooting (log collection and inspection)

  • Making allowed configuration changes

  • Installing supported helper modules (serialization/encryption libraries…)

How to Connect

Connecting by using Remote Desktop is a two-step operation.

  1. Use Remote Desktop to connect to the secure node (HSN01).

  2. Use Remote Desktop on the secure node, to connect to any HDI node in the cluster.

Connection Parameters

When connecting to HSN01 connect by using the Ethernet IP address. If the Ethernet IP is mapped in the corporate DNS, you can use the HSN01 host alias.

When connecting to internal cluster nodes, start Remote Desktop on the HSN01 node. Use the host name or InfiniBand IP address to connect to virtual machines in the cluster.

Specify the host name is in the following format: HDI_region-node_name, for example: H12345-HHN01 or H12345-HDN001.

Specify the login credentials from the appliance domain. (A user name and password were specified when the accounts were created.)

For both Remote Desktop connections, specify the username in the format appliance_domain\john.

Using local machine accounts for Remote Desktop is not recommended.

Permissions

Only members of HDInsight Cluster Administrators group can connect by using Remote Desktop. Regular users are not allowed to remotely connect to region virtual machines.

See Also

Concepts

Introduction (Analytics Platform System)
Understanding the Security Model of the HDInsight Region (Analytics Platform System)


Changing Hadoop Configuration Settings (Analytic Platform System)

[image]

Changing Hadoop Configuration Settings (Analytic Platform System)
See Also Send Feedback

Describes how to change supported Hadoop configuration settings. Use these scenarios to change one or more parameters, revert cluster configurations to factory settings or the last known good configuration, or to restore the Hadoop cluster configuration after a servicing action.

Contents

  • Before You Begin

  • Scenario 1: Change one or more parameters from the list of supported configurations

  • Scenario 2: Revert cluster configurations to factory settings

  • Scenario 3: Revert cluster configurations to the last known good configuration

  • Scenario 4: Restore cluster configuration after servicing action

Before You Begin

Requirements

We only support changing the configuration settings that are listed in Supported Hadoop Configuration Changes (Analytics Platform System).

The scenarios require membership in the HDInsight Cluster Admin security group.

Recommendations

The cluster should not run any workload or data loading during reconfiguration actions. Expect HDInsight to have a period of downtime before new configuration becomes effective. When changing more than one configuration value, it is better to do all changes in a single batch, since changing configuration requires the services to restart. Although it is possible to effectively apply new configurations by restarting only the subset of dependent services that are affected by the change, restarting all Hadoop services is recommended as a more reliable practice.

Updates to HDInsight and servicing actions such as replacing virtual machines can override custom configurations, returning them to their original default values. When make changes, administrators should use HDInsight Configuration Tool (Analytics Platform System) to make sure that last valid version of the cluster configuration is always available. The same tool can be used to revert the cluster to custom configuration settings.

[TOP]

Scenario 1: Change one or more parameters from the list of supported configurations

To change a parameter

  1. Determine if particular configuration needs to be altered on the head node only or on the worker nodes as well.

  2. Determine if particular configuration needs to be altered on the head node only or on the worker nodes as well.

  3. Remotely connect to the head node virtual machine (<HDInsight domain>-HHN01) and edit the targeted configuration files.

  4. If the configuration change is required to be applied across all of the worker nodes repeat step 2 for all data nodes (HDNs) in the cluster.

  5. Restart all Hadoop services. Restarting can be done using the Configuration Manager (dwconfig.exe) or System Center Operations Manager (SCOM).

    To restart using Configuration Manager

    1. Open the Configuration Manager, dwconfig.exe, tool. See Launch the Configuration Manager (Analytics Platform System).

    2. In the Configuration Manager, open the HDInsight Topology menu, select the Service Status page, and click Stop Region. Wait for the operation to complete, and then click Start Region.

    To restart using SCOM

    • In SCOM, go to Microsoft HDInsight, Clusters Diagram, select your appliance cluster, and then click Stop all HDInsight cluster services. Wait for operation to complete, and then click Start all HDInsight cluster services.

  6. After verification, use HDInsight Configuration Tool (Analytics Platform System) to make a backup so that cluster can always be restored to that the latest valid point.

    If cluster doesn’t perform well with new configuration values, use Scenario 2: Revert cluster configurations to factory settings or Scenario 3: Revert cluster configurations to the last known good configuration to revert the cluster to either the factory settings or the last known good configuration.

[TOP]

Scenario 2: Revert cluster configurations to factory settings

The cluster is deployed with the optimal configurations which are safely saved on a backup location, both as part of initial deployment as well as every hotfix action. The default backup location for the configuration files is <OS drive>\HadoopConfBackupRoot\Hadoop\. Configuration files saved this way can be used to revert cluster to the original delivery settings.

To restore the configuration to factory settings

  1. Remotely connect to head node virtual machine (<HDInsight domain >-HHN01) and replace the existing configuration files with the original factory default files which are stored in the folder <OS drive>\HadoopConfBackupRoot\Hadoop\.

  2. If custom configuration included data nodes, the head node and the worker nodes need to be repaired. In this case repeat step 1 for all data nodes in the cluster.

  3. Restart all Hadoop services using the same procedure described in Scenario 1, Step 5.

[TOP]

Scenario 3: Revert cluster configurations to the last known good configuration

Upon every successful configuration change, customers are strongly advised to back up the latest state using HDInsight Configuration Tool (Analytics Platform System).

To restore the cluster configuration

  1. Remotely connect to head node virtual machine (<HDInsight domain >-HHN01) and run HDInsight Configuration Tool (Analytics Platform System) with the –restore switch.

  2. After the script completes successfully, restart all Hadoop services using the same procedure described in Scenario 1, Step 5.

[TOP]

Scenario 4: Restore cluster configuration after servicing action

Any servicing action that affects HDInsight will restore the cluster to the default configuration. If you have applied a custom configuration follow Scenario 3: Revert cluster configurations to the last known good configuration to revert the cluster to the last good known configuration from the backup you have previously made.

Although it is not a common practice, a hotfix servicing option can contain new configuration parameters or new recommended values for existing ones. In that case restoring cluster to custom configuration needs to include manual steps of merging the new and previous configurations. Contact Microsoft Support for help with this scenario.

[TOP]

See Also

Reference

HDInsight Configuration Tool (Analytics Platform System)

Concepts

Supported Hadoop Configuration Changes (Analytics Platform System)
Unsupported HDInsight Actions (Analytics Platform System)


HDInsight Configuration Tool (Analytics Platform System)

[image]

HDInsight Configuration Tool (Analytics Platform System)
Send Feedback

Describes the HDInsightConfigurationUtility PowerShell script that performs backup and restore operations for the HDInsight configuration files used by HDInsight services running in the HDInsight region. Use this script perform configuration backups after every configuration change that you would like to save as a stable restore point, and to restore a configuration backup to an HDInsight region.

Syntax

HDInsightConfigurationUtility.ps1
    { –backup | -restore }
    -UserName
    -Password
    [-force]

Arguments

-backup

Stores a backup of the supported configuration files from all cluster nodes (HHN01 and all HDNs).

-restore

Replace the current configuration files with the latest backup.

-UserName

User name for the administrator account that will run the script.

-Password

Password for the administrator account that will run the script.

-force

Run without showing the confirmation prompt.

This script is designed always to ask for confirmation. This behavior can be overridden by using –force switch, but this pattern of usage is not recommended.

Permissions and Requirements

Requires an account that is a member of the HDInsight Cluster Admins security group.

Requires a remote desktop connection to the <HDInsight Domain>-HHN01 head node in the HDInsight region. For more information, see Connecting to HDInsight Using Remote Desktop (Analytics Platform System).

Error Handling

If the script encounters an error, and your appliance is in a healthy state, try re-running the script.

To further troubleshoot a script error, view the log file called HadoopConfBackup.log, which is located in the same directory as the script.

General Remarks

The PowerShell script HDInsightConfigurationUtility.ps1 is located on the head node (<HDInsight domain name>-HHN01), in folder <OS drive>\HadoopConfBackupRoot.

The HDInsightConfigurationUtility script backups up the configuration files listed for each of these Hadoop services.

  • All services - core-site.xml

  • HDFS Service – hdfs-site.xml

  • MapReduce Service – mapred-site.xml

  • Hive Service – hive-site.xml

  • HCatalog, WebHCat Services – webhcat-site.xml, proto-hive-site.xml

  • Oozie Service – core-site.xml, oozie-site.xml, hive.xml

  • Sqoop Service – sqoop-site.xml (including files on worker nodes)

  • Yarn Service – yarn-site.xml

Limitations and Restrictions

Only one backup is supported. Each new backup will replace the previous backup.

Run this script only if HDInsight is in a healthy state and in full capacity (all nodes are running).

Recommendations

We recommend performing a configuration backup after every configuration change that you would like to save as a stable restore point. We strongly recommend that you perform checkpoint.

Examples

Create a backup of the HDInsight supported configuration files

To create a backup of the supported configuration files, open PowerShell and change to the C:\HadoopConfBackupRoot directory. Then run the following command, using an administrator user name.

Copy image Copy Code
PS C:\HadoopConfBackupRoot> .\HDInsightConfigurationUtility.ps1 –backup -<User Name> -Password <password>

The script will prompt you to confirm the action. Enter Y to continue.

As the script runs, it will report progress. When it finishes successfully it outputs this line: SUCCESS: <timestamp>: Backup committed in the database

Restore a backup of the HDInsight supported configuration files

To restore a backup of the supported configuration files, log in to the head node (<HDInsight Domain>-HHN01), open PowerShell and change to the <OS-Drive>:\HadoopConfBackupRoot directory. Run the following command, using an administrator user name.

Copy image Copy Code
PS C:\HadoopConfBackupRoot> .\HDInsightConfigurationUtility.ps1 –restore -<User Name> -Password <password>

The script will prompt you to confirm the action. Enter Y to continue.

As the script runs, it will report progress. When it finishes successfully the last line of output is: SUCCESS: <timestamp>: NodeBackup script executed on machine @{Name=<HDInsight domain>-HDN01; NodeType=Data}


Supported Hadoop Configuration Changes (Analytics Platform System)

[image]

Supported Hadoop Configuration Changes (Analytics Platform System)
See Also Send Feedback

Microsoft only supports changing the following specified (whitelisted) configuration settings. All setting not listed below, should not be changed. Changing items that are not listed, might affect your support contract. For more information about how to change settings, see Changing Hadoop Configuration Settings (Analytic Platform System).

Note Note

HDP 2.0 introduces changes with respect to Hadoop configuration. Some configuration parameters from HDP 1.3 are deprecated in HDP 2.0 and replaced with new parameters while others are not supported (i.e. do not affect system behavior). If you upgraded from previous version of APS and if you had customized configurations you will need to migrate your settings to HDP 2.0 format. For assistance, contact Microsoft Support for help with this scenario.

These configuration files are located in %HADOOP_HOME%/etc/Hadoop.

Core-site.xml

Configuration property | Comment
fs.trash.interval | Number of minutes after which the checkpoint gets deleted. If zero, the trash feature is disabled.
(For AU1 only) fs.checkpoint.period | The number of seconds between two periodic checkpoints.
fs.checkpoint.size | The size of the current edit log (in bytes) that triggers a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
polybase.recursive.traversal | When reading data from external tables, set <polybase.recursive.traversal> to 'false' to read data from only the root folder of the external file location.

Log4J.properties

Configuration property | Comment
hadoop.root.logger | Controls the logging level for hadoop services. It can be changed from WARN to INFO for troubleshooting purposes.

Hdfs-site.xml

Configuration property | Comment
dfs.client.read.shortcircuit.skip.checksum | If this configuration parameter is set, short-circuit local reads will skip checksums. This is normally not recommended, but it may be useful for special setups.
(For AU1) io.bytes.per.checksum (For AU2) dfs.bytes-per-checksum | The number of bytes per checksum. Must not be larger than dfs.stream-buffer-size.
dfs.stream-buffer-size | The size of buffer to stream files. The size of this buffer should probably be a multiple of hardware page size (4096 on Intel x86), and it determines how much data is buffered during read and write operations.
dfs.webhdfs.enabled | Enables WebHDFS (REST API).
dfs.replication | This determines how many copies of each block to store for durability.
dfs.block.size | The size of HDFS blocks. When operating on data stored in HDFS, the split size is generally the size of an HDFS block. Larger numbers provide less task granularity, but also put less strain on the cluster NameNode
dfs.datanode.handler.count | The number of server threads for the datanode.
dfs.namenode.handler.count | The number of server threads for the namenode.
(For AU1) dfs.balance.bandwidthPerSec (For AU2) dfs.datanode.balance.bandwidthPerSec | Specifies the maximum amount of bandwidth that each datanode can utilize for the balancing purpose in term of the number of bytes per second.
(For AU2 only) dfs.namenode.checkpoint.period | The number of seconds between two periodic checkpoints.

Mapred-site.xml

Configuration property | Comment
(For AU1) mapred.map.child.java.opts (For AU2) mapreduce.map.java.opts | This property stores Java options for map tasks.
(For AU1) mapred.reduce.child.java.opts (For AU2) mapreduce.reduce.java.opts | This property stores Java options for to reduce tasks.
(For AU1) mapred.tasktracker.map.tasks.maximum (For AU2) mapreduce.tasktracker.map.tasks.maximum | The maximum number of map task slots to run simultaneously.
(For AU1) mapred.tasktracker.reduce.tasks.maximum (For AU2) mapreduce.tasktracker.reduce.tasks.maximum | The maximum number of reduce task slots to run simultaneously.
(For AU1) io.sort.mb (For AU2) mapreduce.task.io.sort.mb | This value sets the size, in megabytes, of the memory buffer that holds map outputs before writing the final map outputs. Lower values for this property increases the chance of spills.
(For AU1 only) io.sort.record.percent (For AU2) | The percentage of the memory buffer specified by the io.sort.mb property that is dedicated to tracking record boundaries.
(For AU1) io.sort.spill.percent (For AU2) mapreduce.map.sort.spill.percent | This property's value sets the soft limit for either the buffer or record collection buffers.
(For AU1) mapred.tasktracker.expiry.interval (For AU2) mapreduce.jobtracker.expire.trackers.interval | This property's value specifies a time interval in milliseconds. After this interval expires without any heartbeats sent, a TaskTracker is marked lost.
(For AU1) mapred.map.tasks.speculative.execution (For AU2) mapreduce.map.speculative | If true, then multiple instances of some map tasks may be executed in parallel.
(For AU1) mapred.reduce.tasks.speculative.execution (For AU2) mapreduce.reduce.speculative | If true, then multiple instances of some reduce tasks may be executed in parallel.
(For AU1) mapred.map.tasks (For AU2) mapreduce.job.maps | The default number of map tasks per job. Ignored when the value of the mapred.job.tracker property is local.
(For AU1) mapred.reduce.tasks (For AU2) mapreduce.job.reduces | The default number of reduce tasks per job. Ignored when the value of the mapred.job.tracker property is local.
(For AU1 only) tasktracker.http.threads (For AU2) mapreduce.tasktracker.http.threads | The number of worker threads that for the HTTP server. This is used for map output fetching.
(For AU1) mapred.reduce.parallel.copies (For AU2) mapreduce.reduce.shuffle.parallelcopies | The default number of parallel transfers run by reduce during the copy (shuffle) phase.
(For AU1) mapred.reduce.slowstart.completed.maps (For AU2) mapreduce.job.reduce.slowstart.completedmaps | Fraction of the number of maps in the job which should be complete before reduces are scheduled for the job.
(For AU1) mapred.min.split.size (For AU2) mapreduce.input.fileinputformat.split.minsize | The minimum size chunk that map input should be split into. File formats with minimum split sizes take priority over this setting.
(For AU1) mapred.job.tracker.handler.count (For AU2) mapreduce.jobtracker.handler.count | This property's value sets the number of server threads for the JobTracker.

Oozie-site.xml

Configuration property | Comment
oozie.service.coord.normal.default.timeout | Default timeout for a coordinator action input check (in minutes) for normal job.
oozie.command.default.lock.timeout | Default timeout (in milliseconds) for commands for acquiring an exclusive lock on an entity.
oozie.notification.url.connection.timeout | Defines the timeout, in milliseconds, for Oozie HTTP notification callbacks. Oozie does HTTP notifications for workflow jobs which set the 'oozie.wf.action.notification.url', 'oozie.wf.worklfow.notification.url' and/or 'oozie.coord.action.notification.url' properties in their job properties.
use.system.libpath.for.mapreduce.and.pig.jobs | If set to true, submissions of MapReduce and Pig jobs will include automatically the system library path, thus not requiring users to specify where the Pig JAR files are. Instead, the ones from the system library path are used.
oozie.service.UUIDService.generator | random : generated UUIDs will be random strings. counter: generated UUIDs generated will be a counter postfixed with the system startup time.
oozie.action.hadoop.delete.hdfs.tmp.dir | If set to true, it will delete temporary directory at the end of execution of map reduce action.
oozie.action.pig.delete.hdfs.tmp.dir | If set to true, it will delete temporary directory at the end of execution of pig action.
oozie.service.LiteWorkflowStoreService.user.retry.max | Automatic retry max count for workflow action is 3 in default.
oozie.service.LiteWorkflowStoreService.user.retry.inteval | Automatic retry interval for workflow action is in minutes and the default value is 10 minutes.
oozie.service.LiteWorkflowStoreService.user.retry.error.code | Automatic retry interval for workflow action is handled for these specified error code: FS009, FS008 is file exists error when using chmod in fs action. JA018 is output directory exists error in workflow map-reduce action. JA019 is error while executing distcp action. JA017 is job not exists error in action executor. JA008 is FileNotFoundException in action executor. JA009 is IOException in action executor.

yarn-site.xml

The HDInsight region uses the default settings. Changing these settings is not supported.

See Also

Concepts

Changing Hadoop Configuration Settings (Analytic Platform System)
Unsupported HDInsight Actions (Analytics Platform System)


Unsupported HDInsight Actions (Analytics Platform System)

[image]

Unsupported HDInsight Actions (Analytics Platform System)
See Also Send Feedback

When Analytics Platform System tools are available, always use Analytics Platform System tools to perform actions. The Analytics Platform System tools are engineered to perform actions in methods that keep Analytics Platform System running reliably.

Unsupported Actions

Unsupported Action | Recommended Action
Do not start or stop services from the Windows local services tool. | Use the Services Status page of the Configuration Manager tool. For more information, see HDInsight Services Status (Analytics Platform System).
Do not manage user accounts in Active Directory. | Use the User Management page of the Configuration Manager tool. For more information, see . HDInsight User Management (Analytics Platform System)
Do not enable remote desktop access to regular users. | Regular users should access HDInsight and Hadoop services through the Developer Dashboard tool. For more information, see . HDInsight Region Developer Dashboard (Analytics Platform System).
Do not manage IIS hosted apps or change IIS configuration. | The current configuration is required for Analytics Platform System.
Do not change SQL Server configuration on HMN01. | The current configuration is required for Analytics Platform System.
Do not use Hadoop command line options to run Hadoop services. | Use the DW Configuration Manager to start and stop HDInsight services. For more information, see HDInsight Services Status (Analytics Platform System).
Do not change Hadoop configurations that are not specified in the topic Supported Hadoop Configuration Changes (Analytics Platform System). | Not applicable.
Do not install additional tools on APS. For example installing HBase, Flume, Mahout, or ZooKeeper in the HDInsight region is not supported. | Not applicable.

See Also

Concepts

Changing Hadoop Configuration Settings (Analytic Platform System)
Supported Hadoop Configuration Changes (Analytics Platform System)


Appliance Management Tasks (Analytics Platform System)

[image]

Appliance Management Tasks (Analytics Platform System)
Send Feedback

This guide describes how to perform appliance administration tasks for SQL Server PDW.

Guide Sections

  • Appliance Installation and Configuration Overview (Analytics Platform System)

  • PDW Domain Security (SQL Server PDW)

  • Connect to Appliance Nodes (Analytics Platform System)

  • Antivirus Software (Analytics Platform System)

  • Appliance Configuration (Analytics Platform System)

  • Software Servicing (Analytics Platform System)

  • PDW Services Status (Analytics Platform System)

  • Power the APS Appliance On or Off (Analytics Platform System)

  • Appliance Monitoring (Analytics Platform System)


Appliance Installation and Configuration Overview (Analytics Platform System)

[image]

Appliance Installation and Configuration Overview (Analytics Platform System)
Send Feedback

Walks SQL Server PDW appliance administrators through the initial steps to set up and get started using your new SQL Server PDW appliance.

Contents

  • Before You Begin

  • 1. Install the Hardware

  • 2. Power On the Appliance

  • 3. Configure the Appliance

  • 4. Set Up Software Servicing

  • Next Steps

Before You Begin

Before you begin to install, configure, and use your new appliance, we recommend reviewing information about the appliance components. Review the following to familiarize yourself with the appliance:

  • Review Understanding the Appliance Nodes and Hardware (SQL Server PDW) to be sure you understand the components included in your new appliance.

  • Review Client Tools (SQL Server PDW) to understand how and when appliance administrators will connect to each appliance node.

1. Install the Hardware

Your new appliance will be delivered on pallets to the dock at your data center.

Important note Important

In some cases, your IHV will unpack, rack, and connect the appliance for you in your data center. If so, these instructions are not necessary and you can skip to the Configure Appliance section below.

If your IHV is not performing the hardware install, use the following steps to install the hardware.

Task | Description
Confirm documentation | Confirm that you have received all necessary documents and information from your independent hardware vendor (IHV). See Information to Obtain from Your IHV (Analytics Platform System).
Install hardware | Verify the data center can accommodate the appliance. Move the appliance components to the data center. Rack the network switches, PDUs, and cabling. See Hardware Installation (Analytics Platform System).

[TOP]

2. Power On the Appliance

Task | Description
Power on the appliance | Power on each appliance component node in the required order, waiting as necessary to confirm that no errors are encountered.

[TOP]

3. Configure the Appliance

Task | Description
Configure the appliance by using the SQL Server PDW Configuration Manager | Use the Configuration Manager to set appliance passwords, time zones, network and firewall settings, security certificates, and performance and other settings on your appliance. See Appliance Configuration (Analytics Platform System).
Caution note Caution

Configuration changes should only be made using the SQL Server PDW Configuration Manager. Changes not exposed through Configuration Manager are not supported. For example, the SQL Server PDW appliance only supports the US-English language setting.

[TOP]

4. Set Up Software Servicing

Task | Description
Apply SQL Server PDW updates | (Optional) You may need to apply one or more SQL Server PDW updates to update your SQL Server PDW software to the latest version. See Apply Analytics Platform System Hotfixes (Analytics Platform System).
Configure Windows Server Update Services | Configure the appliance to receive updates from Windows Server Update Services for supporting software. See Download and Apply Microsoft Updates (Analytics Platform System).

[TOP]

Next Steps

After you have completed all of the preceding steps, your appliance is ready for use. You or other personnel at your location can proceed with the following tasks.

Task | Description
Install SQL Server PDW drivers and configure connectivity | Configure local computers to connect to SQL Server PDW by using SQL Server Data Tools, sqlcmd, business intelligence software, or other tools. See Client Tools (SQL Server PDW).
Create logons and server roles, and assign permissions | Plan and create logons and server roles that will allow users to log on to SQL Server PDW with the appropriate permissions. See PDW Permissions (SQL Server PDW).
Configure the Azure Data Management Gateway | The Gateway enables Azure users to access on-premises APS data by exposing APS data as secure OData feeds. The Gateway is installed on the Control node already. Ask Microsoft for assistance with configuration.
Monitor queries and appliance users | Use the Admin Console and other resources to monitor the queries and appliance users. See Monitor the Appliance by Using the Admin Console (Analytics Platform System) and User Sessions (SQL Server PDW).
Load data to SQL Server PDW | Load data to your appliance. See Load (SQL Server PDW).
Create a disaster recovery plan | Plan how you will protect your data from hardware failures or data overwrites. Create a plan using regular backups and restore plans in case of data corruption or loss. See Create a Disaster Recovery Plan (SQL Server PDW).
Monitor the appliance | Monitor the appliance state, health, and performance by using system views, logs, and the Admin Console. Correct or report any problems. See Monitor Appliance Health State (Analytics Platform System) and sys.dm_pdw_component_health_status (SQL Server PDW).

[TOP]


Information to Obtain from Your IHV (Analytics Platform System)

[image]

Information to Obtain from Your IHV (Analytics Platform System)
Send Feedback

When your independent hardware vendor (IHV) delivers your new SQL Server PDW appliance to you, they will also deliver information on the appliance hardware and the configuration they have performed on your appliance. You will need this information to administer your appliance.

The following list shows information that is usually needed from your IHV. In some cases, additional or other information is needed. Check with your IHV to make sure all relevant information has been transferred to you with the appliance delivery.

Information or Document

Description

Bill of Materials (BOM)

Your Bill of Materials lists the components that are included in your appliance. This information is necessary to confirm that all components were delivered.

Important note Important

Your Bill of Materials should include weights for each of the appliance nodes and for each complete rack. This information is important when planning how to handle and move appliance components, and for making sure your data center can accommodate the appliance. If your BOM does not include node weights, make sure to obtain this information from your IHV for all nodes.

Cabling diagrams

Cabling diagrams show how to connect the network, power, and other cables for each appliance rack. These diagrams are needed when installing the appliance in your data center, and any time you need to remove or replace a component.

Appliance racking requirements

Before your appliance can be installed in your data center, you need to know if your data center meets the airflow and cable-length requirements for the appliance, as well as the size and power requirements for the components. See also Bill of Materials (BOM) above for information about appliance component weights, which is also required.


Hardware Installation (Analytics Platform System)

[image]

Hardware Installation (Analytics Platform System)
See Also Send Feedback

This topic describes how to move, unpack, and install the hardware for your SQL Server PDW appliance. This topic is informational only and is intended to help you understand the process. Your appliance should be unpacked, installed, and verified before it is turned over to you. Customer participation is required for items such as data center access, electrical power, and Ethernet connections.

Contents

  • Before You Move Any Components from the Loading Dock

  • Move the Racks from the Loading Dock into the Data Center

  • Install Additional Components

Before You Move Any Components from the Loading Dock

Perform the following tasks before you move, unpack, or rack any of the appliance components.

Task

Description

Verify that all components have arrived

Use the Bill of Materials (BOM) to verify that all components have arrived and are on their pallets at the receiving dock for your data center.

Verify that the data center meets all requirements for the appliance

Start this task by reviewing the hardware specifications and cabling diagrams providing by your IHV. The next steps provide specifics about rack space and connectivity requirements.

Verify that the data center has proper rack space

  • Verify that the data center has enough space for all of the appliance racks.

  • Verify that the rack space is empty and ready to receive the appliance racks.

Verify that the data center meets connectivity requirements

  • Verify that the data center meets the cabling requirements in the cabling diagrams.

  • Verify that there will be room for all cables and power cords after the appliance nodes are racked.

Verify that the floors between the dock and the racks meet weight requirements

  • Verify that all flooring between the pallets and the racks can support the weight of the appliance nodes, especially in data centers with raised floors.

  • Contact your IHV for information on the weight of each component.

Secure the data center rack

Secure the data center rack in place using additional equipment as required for your data center location, such as earthquake straps in geographic areas prone to earthquakes.

Prepare for assistance with transporting the components

Determine in advance what assistance, equipment, and tools you will need to handle each component safely and without causing damage.

TOP

Move the Racks from the Loading Dock into the Data Center

Each pallet contains all components for one appliance rack, including nodes, cables, cords, etc.

Use the following checklist to move each appliance rack from the pallet at the loading dock to its rack location in the data center. Move the control rack first, and then move the other appliance data racks.

Caution note Caution

Failure to perform these steps exactly as described could result in bodily harm, damage to your SQL Server PDW appliance, or other problems.

Never attempt to lift or move an appliance node or other heavy component without assistance or proper equipment. Contact your IHV for information on the weight of each component so that you can determine in advance what assistance, equipment, and tools you will need to handle each component safely and without causing damage.

Task | Description
Verify that the pallet is level | Before you begin to move or unpack the pallet, be sure it is on level ground.
Unbolt a node from the pallet | Starting at the top of the pallet, unbolt the top node from the pallet.
Move the node to a dolly or cart that can support the weight | Use ramps and proper lifting/moving techniques to move the node to a dolly or cart that can support the weight.
Transport the node into the data center | Use proper lifting/moving techniques to move the node into position in the data center rack.
Secure the node in the data center rack | Secure the node in place in the data center rack.
Repeat these steps for the next node or component | Repeat these steps to move the next node or other appliance component into the data center.

TOP

Install Additional Components

Use the following checklist to install the additional components.

Task | Description
Unpack and rack network switches and PDUs | Use the rack diagrams to place the network switches and PDUs in the proper location in the rack.
Connect the Infiniband and Ethernet cables according to the cable labels | See the cabling diagram. Each cable has a label on each end that specifies where it needs to be connected.
Connect all power cables | See the cabling diagram.
Turn on the power supply to the racks and the PDUs | Connect the power supply to the racks and from the racks to the PDUs. Do not power on any of the other appliance components at this time.

TOP

See Also


Connect to Appliance Nodes (Analytics Platform System)

[image]

Connect to Appliance Nodes (Analytics Platform System)
See Also Send Feedback

This topic explains the various ways to connect to each node in the Analytics Platform System appliance.

Connecting with Hadoop

Before using Hadoop with SQL Server PDW, ask your appliance administrator to install the Java Runtime Environment onto SQL Server PDW. For instructions, see Configure PolyBase Connectivity to External Data (Analytics Platform System) in the Appliance Operations Guide.

Connecting to Appliance Nodes

Each of the appliance nodes is accessed directly only under specific usage scenarios and by specific user types. The following table lists each appliance node and the scenarios under which users will connect directly to that node.

For information on the purpose of each node, see Understanding SQL Server PDW (SQL Server PDW).

Node

Access Scenarios

Control node

Use a web browser to access the Admin Console, which runs on the Control node. For more information, see Monitor the Appliance by Using the Admin Console (Analytics Platform System).

All client applications and tools connect to the Control node, regardless of whether the connection is using Ethernet or InfiniBand.

  • To configure an Ethernet connection to the Control node, use the Control node Cluster IP address and port 17001. For example, "xxx.xxx.xxx.xxx,17001".

  • To configure an InfiniBand connection to the Control node, use appliance_domain-SQLCTL01 and port 17001. By using appliance_domain-SQLCTL01, the appliance DNS server will connect your server to the active InfiniBand network. To configure your non-appliance server to use this, see Configure InfiniBand Network Adapters (SQL Server PDW).

The appliance administrator connects to the Control node to perform management operations. For example, the appliance administrator performs the following operations from the Control node:

  • Configure Analytics Platform System with the dwconfig.exe configuration tool.

Compute node

Compute node connections are directed by the Control node. The IP addresses of Compute nodes are never entered into application commands as parameters.

For loading, backup, Remote Table Copy, and Hadoop, SQL Server PDW does send or receive data directly in parallel between the Compute nodes and the non-appliance nodes or servers. These applications connect with SQL Server PDW by connecting to the Control node, and then the Control node directs SQL Server PDW to establish communication between the Compute nodes and the non-appliance server.

For example, these data transfer operations happen in parallel with direct connections to the Compute nodes:

  • Loading from the loading server to SQL Server PDW.

  • Backing up a database from SQL Server PDW to the backup server.

  • Restoring a database from the backup server to SQL Server PDW.

  • Querying Hadoop data from SQL Server PDW.

  • Exporting data from SQL Server PDW to an external Hadoop table.

  • Copying a SQL Server PDW table to a remote SMP SQL Server database.

[TOP]

Connecting to the Ethernet and InfiniBand Networks

Remote servers can connect through either the appliance InfiniBand network or through the Ethernet network. For performance reasons, loading servers, backup servers, and servers receiving data via CREATE REMOTE TABLE AS SELECT statements, usually transfer data through the appliance InfiniBand network.

You can configure non-appliance servers to belong to your own customer workgroup or domain, and then use your own customer network to connect to those servers and transfer data to them. Also, non-appliance servers connected to the appliance InfiniBand have the option to transfer data to each other over the appliance InfiniBand network; be careful with this because it can slow the performance of SQL Server PDW.

For example, this statement adds network credentials for performing backups through InfiniBand to a backup server that has an InfiniBand IP address 172.16.14.254. The appliance name is mypdw, and the statement is performed from the backup server. Before running this statement, you need to fill in your own parameters.

Copy image Copy Code
sqlcmd -S "mypdw-sqlctl01,17001" -U sa -P password -I -Q "exec sp_pdw_add_network_credentials '172.16.14.254', 'appliance_domain\Administrator', 'password'"

For example, a loading command will start with the following:

Copy image Copy Code
dwloader –S appliance_domain-SQLCTL01

See Also

Concepts

Configure an External Windows System To Receive Remote Table Copies Using InfiniBand (SQL Server PDW)


Antivirus Software (Analytics Platform System)

[image]

Antivirus Software (Analytics Platform System)
See Also Send Feedback

If your data center requires antivirus software, use these guidelines to install antivirus software on Analytics Platform System. We recommend not installing antivirus software unless it is a firm requirement of your data center.

Caution note Caution

We strongly recommend that you individually assess the security risk for each computer and for Analytics Platform System as a whole, and that you select the tools that are appropriate for the security risk level of each computer. Additionally, we recommend that before you roll out any virus-protection project, you test the whole system under a full load to measure any changes in stability and performance.

Virus protection software requires some system resources to execute. You must perform testing before and after you install your antivirus software to determine whether there is any performance effect on the Analytics Platform System.

This topic is based on the guidance in How to choose antivirus software to run on computers that are running SQL Server and KB Article 961804.

Exclusion List for Physical Hosts

To install the antivirus software on the physical hosts, exclude the following list of directories and processes. These should not be scanned by the antivirus software.

Exclude these directories:

  • C:\ProgramData\Microsoft\Windows\Hyper-V - Virtual machine configuration directory

  • C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks - Default virtual hard disk drive directory

  • C:\clusterStorage - Virtual hard disk drive directories

Exclude these processes:

  • Virtual machine management (Vmms.exe)

  • Virtual machine work processes (Vmwp.exe)

Exclusion List for Virtual Machines (VMs)

To install the antivirus software on the VMs, exclude the following list of directories and files. These should not be scanned by the antivirus software.

PDW_region-CTL01

  • C:\windows\cluster\

  • G:\

appliance_domain-AD01 and appliance_domain-AD02

  • No restrictions

Compute node VMs

  • C:\windows\cluster\

  • G:\

appliance_domain-VMM

  • No restrictions

appliance_domain-WDS

  • No restrictions

appliance_domain-ISCSI01

  • C:\iscsitarget

See Also

Concepts

Appliance Management Tasks (Analytics Platform System)


Appliance Configuration (Analytics Platform System)

[image]

Appliance Configuration (Analytics Platform System)
See Also Send Feedback

Provides checklists for the tasks required to configure Analytics Platform System for your own environment. These configuration tasks are necessary before you can use the appliance.

Caution note Caution

Using Analytics Platform System Configuration Manager is the best way, and the only supported way, to perform the tasks available in the tool.

Contents

  1. Before You Begin

  2. Perform Configuration Manager Tasks

  3. Perform Additional Configuration Tasks

[TOP]

Before You Begin

Prerequisites

  1. The appliance must be installed in the data center and powered on.

  2. Make sure you have the following information, which is provided by your IHV:

    • External IP address for the PDW Control node (PDW_region-CTL01)

    • Appliance domain name

    • User name and password for the appliance domain administrator

  3. Obtain a trusted certificate. You will import this in a later step to allow clients to connect to the Admin Console with secure connections. Save the certificate to the Control node in a password protected PFX file.

  4. Launch the Configuration Manager, using the following steps:

    1. Use Remote Desktop to connect to the PDW Control node (PDW_region-CTL01). (You may need to connect with the external IP address for CTL01.)

    2. Launch the Configuration Manager from the Start menu of the PDW Control node. The first screen of the Configuration Manager displays the appliance topology, which was created by your IHV. It is a list of the hardware nodes recognized by your SQL Server PDW software as part of your appliance. You should not need to change any settings on the Appliance Topology screen.

[TOP]

Perform Configuration Manager Tasks

The SQL Server PDW Configuration Manager (PDWCM) is an appliance administration tool that SQL Server PDW system administrators use to perform appliance-level operations and to change appliance-level settings. For example, use PDWCM to reset passwords, set the time zone, change IP addresses, configure SSL certificates, enable remote access through the firewall, start or stop the appliance, and set Instant File Initialization.

Use Configuration Manager to perform the following configuration tasks.

Configuration Task

Description

Become familiar with the physical component names

PDW and Appliance Fabric Physical Components (Analytics Platform System)

Launch SQL Server PDW Configuration Manager

Launch the Configuration Manager (Analytics Platform System)

Change the domain administrator password

The appliance has a private Windows Active Directory Domain Services that is used to authenticate nodes within the appliance.

Your IHV set up the appliance with a default domain administrator password. This needs to be changed to a password that is secure.

The Configuration Manager is the only supported way to change the domain administrator password.

For more information, see Password Reset (Analytics Platform System).

Change the password for the sa logon

SQL Server PDW has a system administrator logon named sa. The sa logon has all privileges. It can grant, deny, or revoke any permission. It can also see all system views.

For more information, see Password Reset (Analytics Platform System).

Set the appliance time zone

Set the time (local or other desired time) for all appliance nodes.

For more information, see Appliance Time Zone Configuration (Analytics Platform System).

Specify externally facing network settings for the SQL Server PDW appliance

Appliance Network Configuration (Analytics Platform System)

Import a security certificate for the Admin Console

A certificate can provide Secure Sockets Layer (SSL) connections over HTTPS to the Monitor the Appliance by Using the Admin Console (Analytics Platform System). By default, the Admin Console includes a self-signed certificate that provides privacy, but not server authentication. This certificate also returns an error in Internet Explorer stating: “There is a problem with this website's security certificate” when the user connects. Although this connection encrypts in-flight data between the client and the server, the connection is still at risk from attackers.

SQL Server PDW administrators should immediately acquire a certificate that chains to a trusted certificate authority recognized by clients, in order to have a secure connection and remove the error that Internet Explorer reports. This requires a fully qualified domain name that maps the Control node’s virtual IP Address (recommended) or a certificate name that matches the value that users type into their browser address bars to access the Admin Console.

Use the Configuration Manager to add or remove trusted certificates. Directly using the Microsoft Windows HTTP Services Certificate Configuration Tool (winHttpCertCfg.exe) to manage certificates is unsupported.

For more information, see PDW Certificate Provisioning (Analytics Platform System).

Enable or disable Windows Firewall rules that allow or prevent access to specific ports on the SQL Server PDW appliance.

Your IHV will configure and enable the firewall rules that are necessary for the appliance to operate properly. In most cases you will not enable or disable firewall rules.

For more information, see PDW Firewall Configuration (Analytics Platform System).

Start and stop the SQL Server PDW appliance

Stops or starts the SQL Server PDW appliance. For more information, see PDW Services Status (Analytics Platform System).

Review Instant File Initialization options using the Privileges dialog box

Instant File Initialization is a SQL Server feature that allows data file operations to run more quickly. It is enabled on SQL Server PDW only if the Network Service account has been granted the SE_MANAGE_VOLUME_NAME privilege. It is turned off by default.

For more information, see Instant File Initialization Configuration (Analytics Platform System).

Restore the master database from a backup

Deletes the current master database and replaces it with a backup. For more information, see Restore the Master Database (Analytics Platform System).

[TOP]

Perform Additional Configuration Tasks

After performing the Configuration Manager tasks, perform the following list of additional configuration tasks. Some of these tasks are optional.

Configuration Task

Description

Third-party antivirus software can be installed and configured on the SQL Server PDW appliance for externally facing nodes.

(Optional)

For more information, see Antivirus Software (Analytics Platform System).

The password for DSRM can be changed.

(Optional)

For more information, see Set Admin Password for Logging on to AD Nodes in Directory Services Restore Mode (DSRM) (Analytics Platform System).

Configure the appliance to receive software updates

(Recommended)

The appliance needs to be configured to receive updates to the SQL Server PDW and underlying software.

For more information, see Configure Windows Server Update Services (WSUS) (Analytics Platform System). For information about WSUS, see Software Servicing (Analytics Platform System).

Configure connectivity to external data such as Hadoop or Windows Azure blob storage.

(Optional)

For more information, see Configure PolyBase Connectivity to External Data (Analytics Platform System).

Configure Antivirus Software

(Optional)

Third-party antivirus solutions can be used to protect externally facing nodes but is not required. Follow the guidelines in.

Configure InfiniBand Network Adapters on Backup and Loading Servers

(Optional)

To configure Backup and Loading servers to connect to SQL Server PDW by using the InfiniBand network, you need to configure the network adapters to allow the appliance DNS to resolve the InfiniBand connection to the currently active InfiniBand network.

Configure to send telemetry data to Microsoft

(Optional)

To configure Analytics Platform System to send telemetry data to Microsoft, you need to run a PowerShell script on the Control node. For specific instructions, see Send Telemetry Feedback to Microsoft (SQL Server PDW).

[TOP]

See Also

Concepts

Antivirus Software (Analytics Platform System)
Configure InfiniBand Network Adapters (SQL Server PDW)


PDW and Appliance Fabric Physical Components (Analytics Platform System)

[image]

PDW and Appliance Fabric Physical Components (Analytics Platform System)
See Also Send Feedback

Names and descriptions for the PDW and appliance fabric physical components. The PDW Region contains all of these components.

See also HDInsight Physical Components (Analytics Platform System).

Contents

  • Component Diagrams

  • PDW Components

  • Appliance Fabric Components

Component Diagrams

This shows the names of the physical components and where they are located in the first rack of a 6-Compute node appliance.

PDW Region Component Names - HP

The actual name for PDW components is the PDW Region name, followed by a dash, followed by the component name. For example, if the PDW Region name is PDW123, the actual names are PDW123-CTL01, PDW123-CMP01, etc.

Similarly, the actual name for appliance fabric components is the appliance domain, followed by a dash, followed by the component name. For example, if the appliance domain is FSW123, the appliance fabric VMs are FSW123-WDS, FSW123-AD01, FSW123-VMM, etc.

Here is a consolidated view of a PDW Region with 6 Compute nodes.

PDW Component Names

[TOP]

PDW Components

The PDW virtual machines are part of the PDW Region.

PDW_region-CTL01

A virtual machine that runs the Control node. This runs on HST01 and can fail over to HST02.

Caution note Caution

SQL Server PDW does not support the creation of a snapshot of the CTL01 virtual machine by using Hyper-V Manager. Snapshots rely upon local storage, which will cause errors if the virtual machine attempts to failover to its backup. Creating a snapshot can also cause reliability issues with the other VM's that failover to the passive server.

PDW_region-CMP01 through PDW_Region-CMP06

A virtual machine that runs the Compute node. In this 6-Compute node diagram, the hosts HSA01 through HSA06 run Compute node VMs CMP01 through CMP06 respectively.

[TOP]

Appliance Fabric Components

These components are part of the appliance fabric.

Virtual Machines

appliance_domain-WDS

This virtual machine hosts Windows Deployment Services (WDS), which Analytics Platform System uses deploy Windows operating systems over the appliance network. It also hosts the DHCP service, which allows the appliance hosts to join the appliance network without having a pre-configured IP address.

The appliance_domain-WDS virtual machine runs on HST01 and can fail over to HST02. The WDS virtual machine and the VMM virtual machine, deploy Windows on the physical hosts during the appliance install. During the appliance lifecycle, WDS and VMM perform operations such as replacing a host.

appliance_domain-VMM

The Virtual Machine Manager (VMM) runs in a virtual machine and can fail over to HST02. VMM hosts System Center to deploy the operating system on the physical hosts. The VMM also provides Windows Server Update Services (WSUS) to apply or remove Windows Updates across all of the hosts and virtual machines.

appliance_domain-AD01, appliance_domain-AD02

Active Directory Domain Services, which contains the Domain Name System (DNS), runs in a virtual machine on both HST01 and HST02. For high availability of the appliance, AD01 and AD02 are replicated domain controllers and they do not failover. If one fails, the other one is already available with the correct data.

appliance_domain-ISCSI01

One ISCSI virtual machine runs on each of the hosts with storage attached (HSA01-HSA06). This VM does not failover.

Hosts

appliance_domain-HST01 through appliance_domain-HST06

The hosts for the PDW Control node and appliance fabric virtual machines. HST03 is an optional passive host.

appliance_domain-HSA01 through appliance_domain-HSA08

The hosts with storage attached (HSA). Each HAS host runs one Compute node VM and one ISCSI VM.

Cluster for PDW

appliance_domain-WFOHST01

The PDW cluster is named WFOHST01. It manages all of the physical hosts and virtual machines that belong to PDW.

Direct Attached Storage

appliance_domain-DAS01 through appliance_domain-DAS03

This is the direct attached storage that is connected to the Compute nodes. HP has one DAS for every two Compute nodes. Dell and Quanta have one DAS for every three Compute nodes.

[TOP]

See Also

Concepts

Hardware Configurations (Analytics Platform System)
Appliance Configuration (Analytics Platform System)
Appliance Management Tasks (Analytics Platform System)


HDInsight Physical Components (Analytics Platform System)

[image]

HDInsight Physical Components (Analytics Platform System)
See Also Send Feedback

Names and descriptions for the HDInsight (HDI) Region physical components on the Analytics Platform System appliance.

This topic focuses on the HDI components. For explanations of the PDW and appliance fabric physical components, see PDW and Appliance Fabric Physical Components (Analytics Platform System).

Contents

  • Component Diagrams

  • HDI Components

  • Appliance Fabric Components

Component Diagrams

This shows the names of the physical components and where they are located in the first rack of an appliance. The appliance has both a PDW Region and an HDI Region. The PDW Region has 6 Compute nodes and the HDI Region has 4 Data nodes.

HDInsight Region Component Names

The actual name for HDI components is the HDI Region name, followed by a dash, followed by the component name. For example, if the HDI Region name is HDI123, the actual names are HDI123-HHN01, HDI123-HDN001, etc.

Here is a consolidated view of an HDI Region with 12 Data nodes. It assumes that the appliance uses more than one rack, and that HSA01-HSA06 are being used for PDW Compute nodes.

HDI Region physical component names

[TOP]

HDI Virtual Machines

The HST03 virtual machines can failover to HST04. The Data nodes do not failover.

HDI_region-HHN01

A virtual machine that runs Hadoop master services, libraries and command line tools.

Services: Head node, secondary Name node (HDFS), Job Tracker (Map Reduce), History Server (Map Reduce), Resource Manager (Yarn), Hive Metastore, HiveServer2 (Hive), TempletonService (WebHCat), OozieService(Oozie).

Libraries: Pig, Sqoop

Caution note Caution

Analytics Platform System does not support the creation of a snapshot of the HHN01 virtual machine by using Hyper-V Manager. Snapshots rely upon local storage, which will cause errors if the virtual machine attempts to failover to its backup. Creating a snapshot can also cause reliability issues with the other VM's that failover to the passive server.

HDI_region-HSN01

A virtual machine that runs the secure node gateway which performs authentication and authorization of all incoming external requests. PolyBase requests are not routed to the secure gateway. HSN01 also hosts the HDInsight Region Developer Dashboard.

HDI_region-HMN01

A virtual machine that hosts components used for deployment, monitoring and management of the HDI region. HMN01 also runs SQL Server to store data for management operations and backups of HDI configuration files.

HDI_region-HDN001 through HDI_region-HDN012

A virtual machine that runs these Hadoop distributed processing components: Data Node (HDFS), TaskTracker (Map Reduce), and Node Manager (Yarn).

A virtual machine that run the HDI Data node. In this 12-Data node diagram, the hosts HSA07 through HSA12 run Data node VMs HDN001 through HDN012.

These virtual machines, while configured as roles in the Hadoop Cluster, do not failover since the Hadoop file system (HDFS) already handles replicated copies of the data. If a Data node fails, Hadoop rebuilds the data on another node.

Appliance Fabric Components

These HDI components are part of the appliance fabric.

Hosts

appliance_domain-HST03 through appliance_domain-HST04

The hosts for the HDI Region. The Head node, Secure node, and Management node run on HST03 and can failover to HST04. When an HDI Region is present, the PDW optional passive host is HST05.

appliance_domain-HSA07 through appliance_domain-HSA12

The hosts with storage attached (HSA). Each HSA runs two Data node VMs and one ISCSI VM.

Cluster for HDI

appliance_domain-WFOHST02

The HDI cluster is named WFOHST02. It manages all of the physical hosts and virtual machines that belong to HDI.

Direct Attached Storage

appliance_domain-DAS04 through appliance_domain-DAS06

This is the direct attached storage that is connected to the Data nodes. HP has one DAS for every four Data nodes. Dell and Quanta have one DAS for every six Data nodes.

[TOP]

See Also

Concepts

Hardware Configurations (Analytics Platform System)
Appliance Configuration (Analytics Platform System)
Appliance Management Tasks (Analytics Platform System)


Launch the Configuration Manager (Analytics Platform System)

[image]

Launch the Configuration Manager (Analytics Platform System)
See Also Send Feedback

This topic provides instructions for launching the Configuration Manager for the Analytics Platform System appliance.

Before You Begin

Prerequisites

The Analytics Platform System Configuration Manager can only be run by the appliance domain administrator. To run this tool, you need the password for the appliance domain administrator. To create additional APS administrators, see Create an APS Domain Administrator (APS).

Launch the Configuration Manager Tool

To run the Configuration Manager, use Remote Desktop to connect to the PDW Control node (PDW_region-CTL01)node, and log in as appliance_domain\Administrator. When starting the Configuration Manager program, use the Run as Administrator option to ensure that your administrator credentials are used.

To launch from a browser window

  1. Open a browser and navigate to the directory C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100.

  2. Right-click dwconfig.exe and then click Run as Administrator.

To launch from a command prompt

  1. On the desktop, open the Start menu, click Programs, click Accessories, right-click Command Prompt and then click Run as administrator.

  2. At the command prompt, enter the following command to change directories: cd /d "C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100".

  3. At the command prompt, enter dwconfig.exe.

After the Configuration Manager is started, you will see all available functionality listed in the left pane. The remainder of this section discusses how to perform each action available in the tool.

To close and exit Configuration Manager, click Exit in the lower-right corner of any screen.

See Also

Concepts

Monitor the Appliance by Using the Admin Console (Analytics Platform System)


Appliance Topology (Analytics Platform System)

[image]

Appliance Topology (Analytics Platform System)
See Also Send Feedback

Displays summary information on the entire Analytics Platform System.

Expand the elements of this section to configure information that applies to the entire appliance. Select the Parallel Data Warehouse Topology or HDInsight Topology sections to configure information that applies to only that region of the appliance.

See Also

Concepts

Launch the Configuration Manager (Analytics Platform System)


Password Reset (Analytics Platform System)

[image]

Password Reset (Analytics Platform System)
See Also Send Feedback

The Password Reset page enables you to change the password for the administrator accounts used by Analytics Platform System.

Caution note Caution

Always use the Configuration Manager to update the appliance domain administrator password. Other methods might not update all components of Analytics Platform System and could cause appliance access problems.

You are given the Analytics Platform System passwords when the appliance is delivered. Always change the passwords to new values when you take responsibility for your appliance. There are four passwords to update. The passwords do not have to be the same as each other.

F<xxxx>\Administrator

The Administrator of the appliance domain.

.\Administrator

The local Administrator account on the computers that host the virtual machines.

Important note Important

For appliance update 1, Configuration Manager does not properly change the password of the local administrator accounts throughout the PDW VM's. If this is necessary, contact CSS for additional instructions.

sa

The SQL Server sa login in SQL Server. sa is a member of the sysadmin fixed server role and is a SQL Server administrator. The password of the sa login can also be changed by using the ALTER LOGIN statement.

Password Requirements

Both the domain administrator credentials and the system administrator credentials adhere to the password strength policies for each type of credential. When changing the domain administrator credentials, the new password is updated to the domain where needed throughout the SQL Server PDW.

Important note Important

SQL Server PDW does not support the dollar sign character ($) in the domain administrator or local administrator passwords. The characters ^ % & are permitted in passwords, however PowerShell regards these as special characters. If any of these characters are used in passwords for the system administrator or SQL Server sa accounts (the AdminPassword and PdwSAPassword parameters during setup) then setup, including INSTALL, UPGRADE, REPLACENODE, and PATCHING, will fail. To ensure a successful upgrade when current passwords contain unsupported characters, change these passwords so that they do not contain such characters before running upgrade. After upgrade completes, you can set these passwords back to their original values. For more information about password requirements, see ALTER LOGIN (SQL Server PDW).

To reset a password

  1. Connect to the Control node and launch the Configuration Manager (dwconfig.exe). For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. In the left pane of the Configuration Manager, click Password Reset.

  3. Select the administrator type from the Account drop-down menu, and then enter the new password in the Password and Confirm Password boxes. Click Apply to save your changes.

    Changes you make to these accounts do not affect any currently active sessions, but will be applied at the next logon attempt for each user.

    SQL Server DWConfig Password

See Also

Concepts

Set Admin Password for Logging on to AD Nodes in Directory Services Restore Mode (DSRM) (Analytics Platform System)
Launch the Configuration Manager (Analytics Platform System)


Appliance Time Zone Configuration (Analytics Platform System)

[image]

Appliance Time Zone Configuration (Analytics Platform System)
See Also Send Feedback

The Time Zone page enables you to set the time zone for all nodes on your SQL Server PDW appliance.

To set the time zone

  1. Launch the Configuration Manager. For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. Stop the appliance services by using the Services Status page in the Configuration Manager. See PDW Services Status (Analytics Platform System) for instructions.

  3. In the left pane of the Configuration Manager, click Time Zone. Select the desired time zone from the Time Zone drop-down menu. Depending on your location, you may also choose to select the box next to Automatically adjust clock for Daylight Saving Time.

  4. Click Apply to save your changes.

  5. Restart the appliance services by using the Services Status page in the Configuration Manager. If you are also planning to change the privileges, you can do that before restarting the appliance.

DWConfig Appliance Time

See Also

Concepts

Launch the Configuration Manager (Analytics Platform System)


Appliance Network Configuration (Analytics Platform System)

[image]

Appliance Network Configuration (Analytics Platform System)
See Also Send Feedback

The SQL Server PDW appliance is built and configured with a fix set of IP addresses throughout all servers and applicable devices from the IHV’s factory floor. Upon delivery of the appliance, the external (Ethernet) IP addressed must be reconfigured to match the specific customer’s data center requirements.

Note Note

PDW V1 required 8 IP external (Customer Facing) addresses to provide external connectivity to each of the control rack nodes. PDW 2012 (V2) enhanced network communications by exposing every component of the appliance externally via IP addresses. This approach provides a more robust design which reduces costs, and increases flexibility, and enhances data movement, data loading, and Hadoop integration. The number of IP addresses required depends upon the number of nodes in the appliance and the presence of features such as HDInsight. To accommodate this larger block of IP addresses, the customer should set up a separate subnet for PDW. Within this subnet, there will be sufficient IP address space (up to 250 addresses) to accommodate the components of up to 5 PDW racks.

The Network Configuration page enables you to view the externally facing network settings for the nodes on your Analytics Platform System appliance. This page is read-only.

DWConfig Appliance Network

To update the network configuration on your appliance

Change the IP addresses of the fabric domain, workload domain, and HDInsight domains by editing the AplianceInfo.xml file and then running setup. This is an offline operation. Both the PDW and HDInsight (if present) regions will be automatically stopped during the IP address change.

Note Note

Domain names are provided during setup and are specified as up to 6 alphanumeric characters, starting with a letter. A frequent naming system creates a fabric domain starting with F, a PDW workload domain starting with P, and an HDInsight domain starting with H. This format is presumed throughout the help file topics but is not required. For more information about the domain structure, see PDW Domain Security (SQL Server PDW) and Understanding the Security Model of the HDInsight Region (Analytics Platform System)

To change the IP addresses of the Analytics Platform System

  1. Using the Remote Desktop application, connect to HST01 using the workload domain administrator account.

  2. On the HST01 node, open the appliance info file at c:\pdwinst\media\AplianceInfo.xml.

    Note Note

    If the file is not present, a new file may need to be created.

  3. Update the Ethernet IP values as needed, and save the file.

  4. In a command prompt window, execute the following setup command to update the IP addresses for the PDW region, using the P/F/H domain names and the administrator passwords.

    Copy image Copy Code
    c:\pdwinst\media\setup.exe /action="ConfigureEthernet" /DomainAdminPassword="<password>" /ApplianceInfoFile="C:\PDWINST\media\ApplianceInfo.xml"

Manufacturer References

For additional information about Dell appliances, see:

  • PowerConnect Switch Instructions Dell PowerConnect 6200 Series System CLI Reference Guide

  • iDRAC/BMC Integrated Dell Remote Access Controller 7 (iDRAC7) Version 1.30.30 User's Guide

  • PDU's Dell Metered Rack PDU ftp://ftp.dell.com/Manuals/all-products/esuprt_ser_stor_net/esuprt_rack_infrastructure/dell-metered-pdu-led_User's%20Guide_en-us.pdf

See Also

Concepts

Launch the Configuration Manager (Analytics Platform System)


PDW Topology (Analytics Platform System)

[image]

PDW Topology (Analytics Platform System)
See Also Send Feedback

Displays name and network information about the SQL Server PDW components. Use the Appliance Topology Network Configuration page to change the network IP Addresses.

Expand the elements of this section to configure information that applies to the entire appliance. Select the Parallel Data Warehouse Topology or HDInsight Topology sections to configure information that applies to only that region of the appliance.

DWConfig PDW top menu

See Also

Concepts

Launch the Configuration Manager (Analytics Platform System)


PDW Certificate Provisioning (Analytics Platform System)

[image]

PDW Certificate Provisioning (Analytics Platform System)
See Also Send Feedback

The PDW Certificate Provisioning page of the Analytics Platform System Configuration Manager imports or removes the certificate used by the PDW region. Using, a certificate to encrypt connections can help secure communication to the Control node through SQL Server clients, tools that use the SQL Server PDW drivers, the Admin Console, and Integration Services loads.

Prerequisites

Before installing the certificate, do the following:

  1. Obtain a secure certificate. If you need more information about how to obtain a secure certificate, contact Microsoft Support.

  2. Save the certificate to the Control node in a password-protected PFX file.

For Security Reasons, Obtain a Trusted Certificate

SQL Server PDW supports using a certificate to encrypt connections to the Control node; including connections to the Admin Console.

By default, the Admin Console includes a self-signed certificate that provides privacy, but not server authentication. This can leave communications vulnerable to a man-in-the-middle attack. When a user connects to the Admin Console by using the self-signed certificate, Internet Explorer returns the error: “There is a problem with this website's security certificate”.

Although the connection through the self-signed certificate encrypts in-flight data between the client and the server, the connection is still at risk from attackers.

Caution note Caution

Appliance administrators should immediately acquire a certificate that chains to a trusted certificate authority recognized by clients, in order to have a secure connection and remove the error that Internet Explorer reports.

The certification path must contain the fully qualified domain name that maps to the Control node Cluster IP Address (recommended) or the name that users type into their browser address bars to access the Admin Console.

Use the Analytics Platform System Configuration Manager to add or remove the trusted certificate. Directly using the Microsoft Windows HTTP Services Certificate Configuration Tool (winHttpCertCfg.exe) to manage the certificate is unsupported.

Import or Remove the Certificate

The following instructions show how to import or remove the appliance certificate.

To import the certificate

  1. Launch the Configuration Manager. For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. In the left pane of the Configuration Manager, expand Parallel Data Warehouse Topology, and then click Certificates.

  3. Select Import a certificate and configure the appliance to use it, and then click Browse to browse to and select the certificate file.

  4. Enter the password for the certificate in the Password field.

  5. Click Apply to configure the certificate for the appliance.

SQL Server PDW will not encrypt current connection by using the imported certificate, but will use the certificate for new connections.

To remove the previously imported certificate

  1. Launch the Configuration Manager. For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. In the left pane of the Configuration Manager, expand Parallel Data Warehouse Topology, and then click Certificates.

  3. Select Remove any certificate provisioned in the appliance.

  4. Click Apply to remove the previously imported certificate from the appliance.

SQL Server PDW will continue to encrypt current connections, but will not use the removed certificate for new connections.

DWConfig Appliance PDW Certificate

See Also

Concepts

Launch the Configuration Manager (Analytics Platform System)
HDInsight Certificate Provisioning (Analytics Platform System)


PDW Firewall Configuration (Analytics Platform System)

[image]

PDW Firewall Configuration (Analytics Platform System)
See Also Send Feedback

The Firewall page of the SQL Server PDW Configuration Manager enables you to enable or disable firewall rules that allow or prevent access to specific ports on the Analytics Platform System appliance.

To manage ports and firewall rules for appliance nodes

  1. Launch the Configuration Manager. For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. In the left pane of the Configuration Manager, expand Parallel Data Warehouse Topology, and then click Firewall.

  3. Locate the port or firewall rule to update in the configuration list, and then select or clear the box next to that item. Only SQL Server PDW admin-configurable options are shown in this list, including opening and closing ports on externally facing nodes.

  4. Click Apply to save your changes.

DWConfig Appliance PDW Firewall

External Ports

The following ports are opened for client connections coming from outside of PDW.

Purpose | Port # | Nodes
SQL Client access for PDW (TDS) | 17001 | CTL
Loader Client access (dwloader & SSIS) | 8001 | CTL
Remote desktop access | 3389 | CTL, CMP
SSIS BinaryLoaderDataChannel | 16551 | CTL
dwloader BinaryLoaderDataChannel | 16551 | CMP
SSL encrypted connections (For internal communications, to access the Admin Console, and to access HDInsight cluster services) | 443 | All nodes
SQL Server PDW Load Control Flow - Windows Credentials | 8002 | CTL
_Kerberos | 88 | AD01 and AD02,
_ldap | 389 | AD01 and AD02

Internal Ports

The following ports are used by PDW for internal communication, but are not opened for connections coming from outside of the PDW appliance.

Purpose | Port # | Nodes
DMS Control Channel traffic | 16450 | CTL, CMP
DMS Data Channel traffic | 16550 | CTL, CMP
Internal Diagnostics | 16650 | CTL, CMP
Failover status (DMS) | 15000 | CTL, CMP
Failover status (Engine) | 15001 | CMP
Dynamic (ephemeral) port range | 20000-65535 | CTL, CMP
SQL Server port ranges (TDS) | 1433, 1500-1508 | CTL, CMP
Note Note

Creating external tables or external data sources uses TCP port 8020 by default. These statements can be configured to use other ports instead. The Hortonworks JOB_TRACKER_LOCATION default port is 50300. Integrating with other systems and tools may require additional ports.

See Also

Concepts

HDInsight Firewall Configuration (Analytics Platform System)


PDW Services Status (Analytics Platform System)

[image]

PDW Services Status (Analytics Platform System)
See Also Send Feedback

The Parallel Data Warehouse Services Status page in the Microsoft Analytics Platform System Configuration Manager shows the current status of all SQL Server PDW services, and provides the ability to stop and start the PDW services. This is the only supported method for starting and stopping the PDW services. Note that individual components or services cannot be started independently.

To start or stop the appliance services

  1. To start the appliance services, click Start Appliance.

  2. To stop the appliance services, click Stop Appliance.

It is not necessary to click Apply when starting and stopping the appliance services by using Start Appliance and Stop Appliance.

DWConfig Appliance PDW Services
Note Note

Stopping the PDW Region also stops the PDW agent (sqldwagent) on the nodes of the HDInsight Region. The HDInsight Region is still functional, however health monitoring will not be available. (The PDW agent requires the PDW control node to report health monitoring.)

See Also

Concepts

Power the APS Appliance On or Off (Analytics Platform System)


Instant File Initialization Configuration (Analytics Platform System)

[image]

Instant File Initialization Configuration (Analytics Platform System)
Send Feedback

Instant file initialization is a SQL Server feature that allows data file operations to run more quickly. Checking the box to turn Instant File Initialization on will improve performance of SQL Server PDW. However, if this poses a security risk for you business, then leave the box unchecked.

Security note Security Note

When instant file initialization is enabled, SQL Server does not overwrite deleted bits with zeros. This behavior could create a security vulnerability if unauthorized users gain access to deleted data. However, SQL Server PDW mitigates this risk by ensuring that the SQL Server database and backup files are always attached to an instance of SQL Server; only the SQL Server service account and the local administrator can access the deleted data on SQL Server PDW.

Instant file initialization is not available when TDE is enabled.

Add Permission for the Backup Account

The backup process requires a network credential (Windows user account) that can access the backup storage location. You authorize PDW to use account by using the sp_pdw_add_network_credentials procedure. See BACKUP DATABASE (SQL Server PDW) for the entire backup process. To use instant file initialization, the backup account must be granted the Perform volume maintenance tasks permission.

  1. On the backup server, open the Local Security Policy application (secpol.msc).

  2. In the left pane, expand Local Policies, and then click User Rights Assignment.

  3. In the right pane, double-click Perform volume maintenance tasks.

  4. Click Add User or Group and add any user accounts that are used for backups.

  5. Click Apply, and then close all Local Security Policy dialog boxes.

To Turn Instant File Initialization On or Off

  1. Launch the Configuration Manager. For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. In the left pane of the Configuration Manager, click Instant File Initialization.

  3. To turn on instant file initialization, select the box next to Enable Instant File Initialization on all nodes. To turn off instant file initialization, clear the box next to Enable Instant File Initialization on all nodes.

    Caution note Caution

    When you turn instant file initialization off, the security consideration discussed above for the feature may still apply to files deleted while instant file initialization was enabled.

  4. Click Apply. The change will propagate through the SQL Server instances on SQL Server PDW the next time the appliance services are restarted. To restart the appliance services now, see PDW Services Status (Analytics Platform System).

  5. You might want to repeat the steps described above as Add Permission for the Backup Account to remove the Perform volume maintenance tasks permission.

DWConfig Appliance PDW Instant File Initialization

For more information about instant file initialization, see Instant File Initialization.


Restore the Master Database (Analytics Platform System)

[image]

Restore the Master Database (Analytics Platform System)
Send Feedback

The Restore Master page of the SQL Server PDW Configuration Manager enables you to restore the master database from a backup.

Before You Begin

Security note Security Note

To perform the restore, SQL Server PDW must delete the current master database, which contains user security information and the database catalog. We recommend making a backup of the current master database before performing the restore.

To restore the master database

  1. Launch the Configuration Manager. For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. In the left pane of the Configuration Manager, click Restore Master.

  3. Select the master backup to restore.

  4. Click Apply.

  5. To perform the restore, SQL Server PDW will shut down all appliance services and disconnect all users. After the restore completes, SQL Server PDW will restart the appliance services.

DWConfig Appliance PDW Restore master


HDInsight Topology (Analytics Platform System)

[image]

HDInsight Topology (Analytics Platform System)
See Also Send Feedback

Displays name and network information about the HDInsight components. Use the Appliance Topology Network Configuration page to change the network IP Addresses.

Expand the elements of this section to configure information that applies to the entire appliance. Select the Parallel Data Warehouse Topology or HDInsight Topology sections to configure information that applies to only that region of the appliance.

DWConfig Appliance HDI Top menu

See Also

Concepts

Launch the Configuration Manager (Analytics Platform System)


HDInsight Certificate Provisioning (Analytics Platform System)

[image]

HDInsight Certificate Provisioning (Analytics Platform System)
See Also Send Feedback

The Certificates page of the Analytics Platform System Configuration Manager imports or removes the certificate on the HDInsight region of the Microsoft Analytics Platform System. Using, a certificate to encrypt connections can help secure communication that connect to Analytics Platform System.

Prerequisites

Before installing the certificate, do the following:

  1. Obtain a secure certificate. If you need more information about how to obtain a secure certificate, contact Microsoft Support.

  2. Save the certificate to the Control node in a password-protected PFX file.

For Security Reasons, Obtain a Trusted Certificate

Analytics Platform System supports using a certificate to encrypt connections to HDInsight. Use a certificate to help secure connections to the HDInsight gateway on port 443 and the HDInsight Developer dashboard on port 81. For more information about connecting to the Developer dashboard, see HDInsight Region Developer Dashboard (Analytics Platform System).

If you are using the Microsoft Hive ODBC Driver to connect to HDInsight from BI, analytics, and reporting tools, a trusted certificate is required.

Use the Analytics Platform System Configuration Manager to add or remove the trusted certificate. Directly using the Microsoft Windows HTTP Services Certificate Configuration Tool (winHttpCertCfg.exe) to manage the certificate is unsupported.

Import or Remove the Certificate

The following instructions show how to import or remove the appliance certificate.

To import the certificate

  1. Launch the Configuration Manager. For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. In the left pane of the Configuration Manager, expand HDInsight Topology, and then click Certificates.

  3. Select Import a certificate and configure the appliance to use it, and then click Browse to browse to and select the certificate file.

  4. Enter the password for the certificate in the Password field.

  5. Click Apply to configure the certificate for the appliance.

Analytics Platform System will not encrypt current connection by using the imported certificate, but will use the certificate for new connections.

To remove the previously imported certificate

  1. Launch the Configuration Manager. For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. In the left pane of the Configuration Manager, expand HDInsight Topology, and then click Certificates.

  3. Select Remove any certificate provisioned in the appliance.

  4. Click Apply to remove the previously imported certificate from the appliance.

Analytics Platform System will continue to encrypt current connections, but will not use the removed certificate for new connections.

DWConfig Appliance HDI Certificate

See Also

Concepts

Launch the Configuration Manager (Analytics Platform System)


HDInsight Firewall Configuration (Analytics Platform System)

[image]

HDInsight Firewall Configuration (Analytics Platform System)
See Also Send Feedback

The Firewall page of the SQL Server PDW Configuration Manager enables you to enable or disable firewall rules that allow or prevent access to specific ports on the HDInsight region of the Microsoft Analytics Platform System.

To manage ports and firewall rules for appliance nodes

  1. Launch the Configuration Manager. For more information, see Launch the Configuration Manager (Analytics Platform System).

  2. In the left pane of the Configuration Manager, expand HDInsight Topology, and then click Firewall.

  3. Locate the port or firewall rule to update in the configuration list, and then select or clear the box next to that item. Only administrator configurable options are shown in this list, including opening and closing ports on externally facing nodes.

  4. Click Apply to save your changes.

DWConfig Appliance HDI Firewall

External Ports

The following ports are opened for client connections coming from outside of HDInsight.

Purpose | Port # | Nodes
SSL encrypted connections (For internal communications and to access HDInsight cluster services) | 443 | All nodes
Client access to HDInsight Developer Dashboard | 81 | HSN01
HDInsight remote desktop access | 3389 | HSN01
HDInsight cluster outbound connectivity | Any | HHN01, HDNxyz (all data nodes)
Note Note

Creating external tables or external data sources uses TCP port 8020 by default. These statements can be configured to use other ports instead. The Hortonworks JOB_TRACKER_LOCATION default port is 50300. Integrating with other systems and tools may require additional ports.

See Also

Concepts

PDW Firewall Configuration (Analytics Platform System)


HDInsight Services Status (Analytics Platform System)

[image]

HDInsight Services Status (Analytics Platform System)
See Also Send Feedback

The HDInsight Services Status page in the Microsoft Analytics Platform System Configuration Manager shows the current status of all HDInsight services, and provides the ability to stop and start the HDInsight services. This is the only supported method for starting and stopping the HDInsight services. Note that individual components or services cannot be started independently.

To start or stop the HDInsight services

  1. To start the HDI services, click Start Region.

  2. To stop the HDI services, click Stop Region.

It is not necessary to click Apply when starting and stopping the services by using Start Region and Stop Region.

DWConfig Appliance HDI Services
Note Note

Stopping the PDW Region also stops the PDW agent (sqldwagent) on the nodes of the HDInsight Region. The HDInsight Region is still functional, however health monitoring will not be available. (The PDW agent requires the PDW control node to report health monitoring.)

See Also

Concepts

Power the APS Appliance On or Off (Analytics Platform System)


HDInsight User Management (Analytics Platform System)

[image]

HDInsight User Management (Analytics Platform System)
See Also Send Feedback

Use the HDInsight User Management page in the Microsoft Analytics Platform System Configuration Manager to manage user for the HDInsight Region. For more information about selecting the right type of HDInsight user for your purpose, see the user definitions in the HDInsight Security Model Overview section of Understanding the Security Model of the HDInsight Region (Analytics Platform System).

To manage HDInsight users

  1. Start the Configuration Manager tool.

    For information about starting the Configuration Manager, see Launch the Configuration Manager (Analytics Platform System).

  2. Click HDInsight Topology.

  3. Click User Management.

  4. On the HDInsight User Management page, manage users by clicking Add User, Drop User, or Reset password.

DWConfig Appliance HDI New User

Creating Additional Administrator Accounts

Administrators can create additional user accounts in the HDInsight Cluster Admin role. Members of that role can remotely connect to cluster nodes and use Hadoop command line tools. To gain full permissions in the Hadoop cluster additional steps are needed. The first time a new administrator connects to the server, the new administrator should create a home folder in HDFS. Complete the following steps to create a home folder.

  1. Remotely connect to the head node. This is a two-step process that first connects to the HSN01 node. For more information, see Connecting to HDInsight Using Remote Desktop (Analytics Platform System).

  2. Open the Hadoop command line tool, located at <OS drive>\hadoop-<HDP version>\bin\hadoop.cmd

  3. Issue the following command using your user name: hadoop fs -mkdir /user/<user name>/

See Also

Concepts

Understanding the Security Model of the HDInsight Region (Analytics Platform System)
Appliance Configuration (Analytics Platform System)


Set Admin Password for Logging on to AD Nodes in Directory Services Restore Mode (DSRM) (Analytics Platform System)

[image]

Set Admin Password for Logging on to AD Nodes in Directory Services Restore Mode (DSRM) (Analytics Platform System)
See Also Send Feedback

Directory Services Restore Mode (DSRM) is a boot mode for repairing or recovering Active Directory Domain Services (AD DS). It is used to log on to the appliance AD nodes after AD DS has failed or when AD DS needs to be restored. The password for DSRM was initialized during the appliance setup at the hardware vendor site and should be changed by the appliance administrator. Analytics Platform System has two AD DS (domain controllers); appliance_domain-AD01 and appliance_domain-AD02. For each appliance AD node, change the DSRM password using the following steps.

To reset the administrator password

  1. Open a Command Prompt window on an appliance AD node appliance_domain–ADxx virtual machine.

  2. At the command prompt, type ntdsutil.

  3. At the ntdsutil prompt, type set dsrm password.

  4. At the Reset Administrator Password: prompt, type reset password on server null.

  5. At the prompt, type the new password.

  6. Repeat steps 1 – 5 above for each appliance AD virtual machine.

    Caution note Caution

    Analytics Platform System does not support the dollar sign character ($) in the domain administrator or local administrator passwords. A password containing a dollar sign will validate and be usable but can block upgrade and maintenance activities.

Note Note

If the Active Directory Domain Services or the virtual machine becomes corrupted for a specific AD virtual machine, running ReplaceVM for the affected AD virtual machine is the recommended corrective action. Contact CSS for assistance.

See Also

Concepts

Password Reset (Analytics Platform System)


Configure Windows Server Update Services (WSUS) (Analytics Platform System)

[image]

Configure Windows Server Update Services (WSUS) (Analytics Platform System)
Send Feedback

These instructions walk you through the steps for using the Windows Server Update Services (WSUS) Configuration Wizard to configure WSUS for Analytics Platform System. You need to configure WSUS before you can apply software updates to the appliance. WSUS is already installed on the VMM virtual machine of the appliance.

For more information about configuring WSUS, see the WSUS Step-by-Step Installation Guide on the WSUS website. After configuring WSUS, see Download and Apply Microsoft Updates (Analytics Platform System) to initiate an update.

Caution note Caution

If you encounter any errors during this configuration process, stop and contact support for assistance. Do not ignore errors or continue in the process after errors are received.

Before You Begin

To configure WSUS, you need to:

  • Have the Analytics Platform System appliance domain administrator account login information.

  • Have a Analytics Platform System login with permissions to access the Admin Console and view appliance state information.

  • Know the IP address of the upstream WSUS server if you are planning to synchronize updates from an upstream WSUS server instead of synchronizing updates directly from Microsoft Update. Make sure your upstream WSUS server is set to allow anonymous connections and supports SSL.

  • Know the IP address of the proxy server if your appliance will be using a proxy server to access the upstream server or Microsoft Update.

  • In most cases, WSUS needs to access servers outside of the appliance. To support this usage scenario the Analytics Platform System DNS can be configured to support an external name forwarder that will allow the Analytics Platform System hosts and Virtual Machines (VMs) to use external DNS servers to resolve names outside of the appliance. For more information, see Use a DNS Forwarder to Resolve Non-Appliance DNS Names (Analytics Platform System).

To configure Windows Server Update Services (WSUS)

  1. Log into the Admin Console. On the Appliance State tab, verify that the Cluster and Network columns show green (or NA) for all nodes. Verify the status indicators for all nodes on the Appliance State.

    • It is safe to continue with green or NA indicators.

    • Evaluate non-critical (yellow) warning errors. In some cases warning messages will not block updates. If there is a non-critical disk volume error that is not on the C:\ drive, you can proceed to the next step before resolving the disk volume error.

    • Most red indicators must be resolved before continuing. If there are disk failures, use the Admin Console Alerts page to verify there is no more than one disk failure within each server or SAN array. If there is no more than one disk failure within each server or SAN array, you can proceed to the next step before fixing the disk failures. Be sure to contact Microsoft support to fix the disk failures as soon as possible.

  2. Log on to the VMM virtual machine as an appliance domain administrator.

  3. Launch the configuration wizard.

    To launch the configuration wizard

    1. In the Server Manager Dashboard, on the Tools menu, click Windows Server Update Services.

    2. In the left pane of the Update Services window, click to expand the Virtual Machine Management node server (appliance_domain-VMM), and then click Options.

    3. In the Options pane, click WSUS Server Configuration Wizard to launch the configuration wizard.

      Server Manager Dashboard menu

    4. If this is the first time you have run the WSUS wizard, you may be asked to configure a directory for storing the updates. C:\wsus is an appropriate location; however you may provide a different path.

    5. Review the Before You Begin list of items to complete before you complete the wizard.

      WSUS Before You Begin

    6. On the Join the Microsoft Update Improvement Program page, select Yes, I would like to join the Microsoft Update Improvement Program, and then click Next.

      WSUS Improvement Program

    You should now see the Choose Upstream Server page. The following screenshot is the starting point of the configuration wizard.

    WSUS Upstream Server Sync

  4. Choose the upstream server.

    On the Choose Upstream Server page of the WSUS configuration wizard, you will select how WSUS on the Virtual Machine Management node will connect to an upstream server to obtain software updates. Your two choices are to synchronize the upstream server with Microsoft Update or to synchronize updates with another Windows Server Update Services server.

    To update by using Microsoft Update

    • If you choose to synchronize with Microsoft Update, you do not need to make any changes to the Choose Upstream Server page. Click Next.

      WSUS Upstream Server Sync

    To update from another WSUS server

    1. If you choose to synchronize with a source other than Microsoft Update (an upstream server), specify the server (enter the IP address) and the port on which this server will communicate with the upstream server.

      WSUS Upstream Server Sync from WSUS

    2. To use Secure Sockets Layer (SSL), select the Use SSL when synchronizing update information check box. In that case the servers will use port 443 for synchronization.

      WSUS Upstream Server Sync from WSUS SSL

    3. If this is a replica server, select the This is a replica of the upstream server check box. It is possible to select both Use SSL when synchronizing update information and This is a replica of the upstream server.

      WSUS Upstream Server Replica

    4. At this point, you are finished with upstream server configuration. Click Next, or select Specify proxy server in the left navigation pane.

  5. Specify the proxy server.

    If this server requires a proxy server to access Microsoft Update or a different upstream server, you can configure the proxy server settings here; otherwise, click Next.

    To configure proxy server settings

    1. On the Specify Proxy Server page of the configuration wizard, select the Use a proxy server when synchronizing check box, and then type the proxy server IP address (not name) and port number (port 80 by default) in the corresponding boxes.

    2. If you want to connect to the proxy server by using specific user credentials, select the Use user credentials to connect to the proxy server check box, and then type the user name, domain, and password of the user in the corresponding boxes. If you want to enable basic authentication for the user connecting to the proxy server, select the Allow basic authentication (password is sent in cleartext) check box.

      WSUS Proxy Credentials

    3. At this point, you are finished with proxy server configuration. Click Next to go to the next page, where you can start to set up the synchronization process.

  6. Start connecting.

    WSUS Proxy Start Connecting

    Click Start Connecting.

    After the connection has succeeded, click Next to go to the next page, where you can choose languages.

  7. Choose languages.

    Select Download updates only in these languages.

    Select English, and then click Next.

  8. Choose products.

    Note Note

    If you are using an Upstream Server, you may not be able to Choose Products. If this option is not available, skip this step.

    Unselect all selected updates.

    Select SQL Server 2014, Windows Server 2012 R2, and System Center 2012 R2 - Virtual Machine Manager, and then click Next.

  9. Choose classifications.

    Note Note

    If you are using an Upstream Server, you may not be able to Choose Classifications. If this option is not available, skip this step.

    Unselect all previously selected updates.

    Select Critical Updates and Security Updates for the updates that will be synchronized for the Analytics Platform System appliance, and then click Next.

  10. Configure the synchronization schedule.

    Select Synchronize Manually, and then click Next.

    Set sync schedule

  11. Begin initial synchronization.

    Select Begin initial synchronization, then click Next.

  12. Finish.

    Click Finish.

Group the Appliance Servers in WSUS

After configuring WSUS for Analytics Platform System, the next step is to group the appliance servers. By adding all of the appliance servers to a group, WSUS will be able to apply software updates to all servers in the appliance.

Note Note

The WSUS system is designed to run asynchronously. Initiating activity does not always result in an immediately update. Therefore, you might need to wait a while until computers will be visible in the WSUS dialog boxes. Running the setup.exe /action=ReportMicrosoftUpdateClientStatus /DomainAdminPassword="<password>" command described at the end of the topic Download and Apply Microsoft Updates (Analytics Platform System) can help refresh the dialog boxes.

To group the appliance servers

  1. Open the WSUS console, right-click All Computers and then click Add Computer Group.

    Add a computer group.

  2. Enter the name “APS” for the computer group, and then click Add.

    Enter name for your new computer group.

  3. Click All Computers again, change the status in the Status drop-down menu to Any, and then click Refresh. You may need to expand All Computers by clicking it on the tree control on the left in order to see the new group you just added.

    Change status to Any and click Refresh.

  4. Select all computers that are part of the appliance, right-click, and then click Change Membership.

    Change membership for all PDW computers.

  5. Select the new computer group that you created by clicking the check box and then clicking OK.

    Set Computer Group Membership

  6. Select the new computer group, change its Status to Any, and then click Refresh. All computers should now be assigned to this group and listed in the right pane. It is generally safe to continue when nodes show warnings such as This node has not reported status yet.

    Change Status to Any and click Refresh.

    [TOP]


Configure PolyBase Connectivity to External Data (Analytics Platform System)

[image]

Configure PolyBase Connectivity to External Data (Analytics Platform System)
See Also Send Feedback

Explains how to configure PolyBase in SQL Server PDW to connect to external Hadoop or Microsoft Azure storage blob data sources. Use PolyBase to run queries that integrate data from multiple sources, including Hadoop, Windows Azure blob storage, and SQL Server PDW.

To configure connectivity

  1. Open a query tool, such as sqlcmd or SQL Server Data Tools (SSDT), and run sp_configure to view the current 'hadoop connectivity' settings.

    hadoop connectivity setting

  2. Decide which Hadoop connectivity setting you need and whether you need to change the current setting. This option applies to the entire SQL Server PDW region. For a full list of the configuration settings and versions, see sp_configure (SQL Server PDW).

  3. To change the 'hadoop connectivity' setting, run sp_configure with RECONFIGURE. Here are some examples.

    Copy image Copy Code
    --Enable connectivity to Hortonworks Data Platform for Windows Server (HDP), HDInsight on Analytics Platform System, or HDInsight’s Microsoft Azure blob storage
EXEC sp_configure 'hadoop connectivity', 4;
RECONFIGURE;

-- Enable connectivity to Hortonworks Data Platform (HDP)for Linux
EXEC sp_configure 'hadoop connectivity', 5;
RECONFIGURE;

-- Enable connectivity to Cloudera CDH for Linux
EXEC sp_configure 'hadoop connectivity', 3;
RECONFIGURE;

-- Disable the Hadoop connectivity option
EXEC sp_configure 'hadoop connectivity', 0;
RECONFIGURE;

    Running sp_configure with RECONFIGURE sets the config value. Restarting the region is needed to set the run value. Since a restart is required after the next stop also, you don’t need to do the restart until after the next step which changes core-site.xml.

  4. To enable Microsoft Azure blob storage as an external data source, add one or more Microsoft Azure storage account access keys to PDW core-site.xml file. To add a key:

    1. Find your Microsoft Azure storage account name. To view your storage accounts, login to Windows Azure Management Portal and click Storage.

      Windows Azure storage account name

    2. Find your Microsoft Azure storage account access key. To do this, click your storage account name, at the bottom of the window click MANAGE ACCESS KEYS. This will show you your account name and storage keys.

      Windows Azure storage account access keys

    3. Open a remote desktop connection to the PDW Control node.

    4. Open the file C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100\Hadoop\conf\core-site.xml.

    5. Add the following property with name and value attributes to core-site.xml.

      Copy image Copy Code
      <property>
  <name>fs.azure.account.key.<your storage account name>.blob.core.windows.net</name>
  <value><your storage account access key></value>
</property>

      This example uses the account name and access key shown previously.

      Copy image Copy Code
      <property>
  <name>fs.azure.account.key.CustomerX.blob.core.windows.net</name>
  <value>yyeTfCxxxxxxxxQ5WdnapXw77W+FwzHUhX/p/f26fIpnNFGtewzyRN90e1/qmTOl1xxxxxxxxa0goG71LsNcw==</value>
</property>
      Caution note Caution

      Take security precautions before storing the access key in core-site.xml. Any user who has CONTROL SERVER or ALTER ANY EXTERNAL DATA SOURCE permission can create an external data source that accesses this account. Once the external data source is created, all SQL Server PDW users who have permission to create tables can create an external table that accesses this storage account. The users will then be able to access account data and consume resources in the account.

    6. Save the changes to core-site.xml.

  5. Add yarn.application.classpath property and values to the yarn-site.xml file.

    Skip this step if you are connecting to an external Hadoop 1.3.

    Beginning with Hadoop 2.0, the yarn-site.xml file contains configuration settings for Hadoop YARN framework. This file is located on the Control node under C:\program files\Microsoft SQL Server Parallel Data Warehouse\100\Hadoop\conf\.

    To run PolyBase queries against an external Hadoop 2.0 Cluster on Windows or Linux, you need to configure the yarn.application.classpath property and values to be consistent with the yarn-site.xml settings on your external Hadoop Cluster. This configuration is necessary even if your external Hadoop Cluster uses the default settings.

    Default settings example:

    Copy image Copy Code
    <property>
    <name>yarn.application.classpath</name>
    <value>
      %HADOOP_CONF_DIR%,
      %HADOOP_COMMON_HOME%/share/hadoop/common/*,
      %HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,
      %HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,
      %HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,
      %HADOOP_YARN_HOME%/share/hadoop/yarn/*,
      %HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*
     </value>
  </property>

    Once any property is defined in yarn-site.xml, PolyBase will use those property settings when it runs queries against the HDInsight Region. If you plan to run PolyBase queries against both the HDInsight Region and an external Hadoop 2.0 Cluster on Windows, there must be consistency among all of the yarn-site.xml files, or else the PolyBase queries will fail.

    To run PolyBase against both the HDInsight Region and an external Hadoop 2.0 Cluster, use the yarn-site.xml default settings on your external Hadoop Cluster.

  6. Restart the PDW region. To do this, use the Configuration Manager tool. See Launch the Configuration Manager (Analytics Platform System).

  7. Verify security settings for Hadoop connections. If the weak authentication on Hadoop side is enabled by using dfs.permission = true, you must create a Hadoop user PDW_User and grant full read and write permissions to this user. SQL Server PDW and the corresponding calls from SQL Server PDW are always issued as PDW_User which is a fixed user name and cannot be changed in this version of Hadoop connectivity and SQL Server PDW release. If security on Hadoop is disabled by using dfs.permission = false, then no further actions need to be taken.

  8. Decide which users can create an external data source to the Microsoft Azure blob storage. Give each of these users the storage account name and also ALTER ANY EXTERNAL DATA SOURCE or CONTROL SERVER permission.

  9. For Hadoop connections, decide which users can create an external data source to Hadoop. Give each of these users the IP address and port number of each Hadoop name node, and give them ALTER ANY EXTERNAL DATA SOURCE OR CONTROL SERVER permission.

  10. Connecting to WASB also requires DNS forwarding to be configured on the appliance. To configure DNS forwarding, see Use a DNS Forwarder to Resolve Non-Appliance DNS Names (Analytics Platform System).

Authorized users can now create external data sources, external file formats, and external tables. They can use these to integrate data from multiple sources including Hadoop, Microsoft Azure blob storage, and SQL Server PDW.

See Also

Concepts

Appliance Configuration (Analytics Platform System)
PolyBase for Hadoop and WASB Integration (SQL Server PDW)


Use a DNS Forwarder to Resolve Non-Appliance DNS Names (Analytics Platform System)

[image]

Use a DNS Forwarder to Resolve Non-Appliance DNS Names (Analytics Platform System)
See Also Send Feedback

A DNS forwarder can be configured on the Active Directory Domain Services nodes (appliance_domain-AD01 and appliance_domain-AD02) of your Analytics Platform System appliance to allow scripts and software applications to access external servers.

Using a DNS Forwarder

The Analytics Platform System appliance is configured to prevent resolving DNS names of servers that are not in the appliance. Some processes, such as Windows Software Update Services (WSUS), will need to access servers outside of the appliance. To support this usage scenario the Analytics Platform System DNS can be configured to support an external name forwarder that will allow Analytics Platform System hosts and Virtual Machines (VMs) to use external DNS servers to resolve names outside of the appliance. Custom configuration of DNS suffixes is not supported, which means you must use fully qualified domain names to resolve a non-appliance server name.

To create a DNS forwarder with the DNS GUI

  1. Log on to the appliance_domain-AD01 node.

  2. Open the DNS Manager (dnsmgmt.msc).

  3. Right-click the name of the server and then click Properties.

  4. On the Advanced tab, unselect the Disable recursion (also disables forwarders) option, and then click Apply.)

  5. Click the Forwarders tab and then click Edit.

  6. Enter the IP address for the external DNS server that will provide the name resolution. The VMs and servers (hosts) in the appliance will connect to external servers by using fully qualified domain names.

  7. Repeat steps 1 – 6 on the appliance_domain-AD02 node

To create a DNS forwarder by using Windows PowerShell

  1. Log on to the appliance_domain-AD01node.

  2. Run the following Windows PowerShell script from the appliance_domain-AD01 node. Before running the Windows PowerShell script, replace IP addresses with the IP addresses of your non-appliance DNS servers.

    Copy image Copy Code
    $DNS=Get-WmiObject -class "MicrosoftDNS_Server"  -Namespace "root\microsoftdns"
$DNS.Forwarders = ("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx")
$DNS.put()

  3. Execute the same command on the appliance_domain-AD01 node.

Configuring DNS resolution for WSUS

SQL Server PDW 2012 provides integrated servicing and patching functionality. SQL Server PDW uses Microsoft Update and other Microsoft servicing technologies. To enable updates the appliance must be able to either connect to a corporate WSUS repository or to the Microsoft public WSUS repository.

For customers that choose to configure the appliance to look for updates on the Microsoft public WSUS repository, the following instructions set the proper configuration details on the appliance.

Note Note

The customer network administrator must provide the IP address for a corporate DNS server that can resolve names at Microsoft.com.

  1. Using remote desktop, log on to VMM VM (<fabric domain>-VMM) using the fabric domain administrator account.

  2. Open the Control Panel, click Network and Internet, and then click Network and Sharing Center.

  3. In the connection list, click VMSEthernet, and then click Properties.

  4. Select Internet Protocol Version 4 (TCP/IPv4), and then click Properties.

  5. In the Alternate DNS server box, add the IP address provided by the customer network administrator.

See Also


Create an APS Domain Administrator (APS)

[image]

Create an APS Domain Administrator (APS)
See Also Send Feedback

Some operations require Analytics Platform System domain administrator privileges. This explains how to create additional appliance domain administrators.

Create a Domain Administrator

To have sufficient permissions to configure all APS nodes, the user running the APS Configuration Manager (dwconfig.exe) must be a member of the Domain Admins group. To start and stop the APS services, the user must be a member of the PdwControlNodeAccess group.

To add a user to the Domain Admins group

  1. Log into the active AD node (appliance_domain-AD01 or appliance_domain-AD02) using an existing appliance domain administrator account.

  2. On the Start menu, click Run. In the Open box, type dsa.msc. Click OK.

  3. In the Active Directory Users and Computers program, right-click Users, point to New, and then click User.

  4. In the New Object – User dialog box, complete the description of the new user, and then click Next.

    Complete the password dialog box, and then click Next.

    Caution note Caution

    SQL Server PDW does not support the dollar sign character ($) in the domain administrator or local administrator passwords. A password containing a dollar sign will valid and be usable but can block upgrade and maintenance activities

    Confirm the new user description, and then click Finish.

  5. In the list of users, double-click the new user to open the user properties dialog box.

  6. On the Member Of tab, click Add.

    Type Domain Admins; PdwControlNodeAccess and then click Check Names. Click OK.

    This adds the new user to the Domain Admins group and the PdwControlNodeAccess group. Click OK.

See Also

Concepts

Launch the Configuration Manager (Analytics Platform System)


Send Telemetry Feedback to Microsoft (SQL Server PDW)

[image]

Send Telemetry Feedback to Microsoft (SQL Server PDW)
See Also Send Feedback

Analytics Platform System has an optional telemetry feature that sends Admin Console data to Microsoft. We encourage you to enable this to help us improve the product.

Note Note

In this release, Microsoft is not actively monitoring the telemetry data. The data is being collected for analysis purposes only.

Contents

  • Privacy

  • Enable Telemetry

  • Disable Telemetry

Privacy

To provide the maximum privacy protection, APS ships without enabling telemetry. Before enabling this feature, first review the Microsoft Analytics Platform System Privacy Statement. Then, to opt-in run the PowerShell script described below.

Enable Telemetry

DNS Forwarding: Sending telemetry data to Microsoft requires Analytics Platform System to connect to the internet through a DNS forwarder. To enable this feature, you must enable DNS forwarding on all hosts and workload VMs. Invoke the Enable-RemoteMonitoring command with the SetupDnsForwarder option to properly configure DNS forwarding and enable telemetry. Invoke the Enable-RemoteMonitoring command without the SetupDnsForwarder option when DNS forwarding is already configured and you only want to enable Heartbeat Monitoring.

Security note Security Note

Enabling DNS forwarding opens the internet connection for all hosts and workload VMs.

To Enable Feedback

  1. Using an appliance domain administrator account, connect to the Control node (appliance_domain-CTL01) and open a command prompt using your Windows administrator credentials.

  2. Navigate to the following directory: C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100.

  3. Import the module Configure-RemoteMonitoring.ps1

    Note Note

    To import you must use two periods in the command.

    Example:

    Copy image Copy Code
    PS C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100> . .\Configure-RemoteMonitoring.ps1

  4. Invoke the Enable-RemoteMonitoring command.

    Note Note

    The script assumes that the internet connection is working properly and does not validate the internet connection.

    1. The first time you enable the telemetry, use the following command to ensure all DNS Forwarders are correctly configured. In this example replace the DNS Forwarded IP address xx.xx.xx.xx with the DNS Forwarder IP address in your environment.

      Copy image Copy Code
      PS C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100> Enable-RemoteMonitoring -SetupDnsForwarder -DnsForwarderIp xx.xx.xx.xx

    2. When DNS forwarders are already configured, such as re-enabling previously disabled telemetry, use the following command to enable telemetry without configuring DNS Forwarding.

      Copy image Copy Code
      PS C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100> Enable-RemoteMonitoring

  5. You will be prompted to read and acknowledge that APS will collect information about the appliance. There will be a link to the APS privacy statement which you can copy and paste into any internet browser for review.

  6. Enter Y to accept and opt in to feedback or enter N to not accept.

  7. If you entered Y there will be a series of commands that will run which will enable the telemetry (and optionally the DNS Forwarder) feature on your appliance. If you see any errors or information that leads you to believe that the command was not successful contact CSS for assistance.

If you entered N, no commands will be run and the feature will not be enabled and there is nothing more for you to do.

There is no harm in running the Enable-RemoteMonitoring command multiple times. If the DNS forwarder is already set you will get a warning message indicating that is the case.

Disable Telemetry

Disabling telemetry will stop all operations which communicate information about the state of the appliance to the APS monitoring service in the cloud.

Important note Important

Performing this operation will not disable your DNS forwarder or internet connection which may be in use by other features in the appliance.

To Disable telemetry

  1. Using an appliance domain administrator account, connect to the Control node (appliance_domain-CTL01) and open a PowerShell window with administrator privileges.

  2. Navigate to the following directory: C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100.

  3. Import the module Configure-RemoteMonitoring.ps1

    Note Note

    To import you must use two periods in the command.

    Example:

    Copy image Copy Code
    PS C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100> . .\Configure-RemoteMonitoring.ps1

  4. Invoke the Disable-RemoteMonitoring command without parameters. This command will stop sending feedback. (This will not affect local monitoring.) However, the command will not disable the DNS Forwarder and/or disable any internet connectivity. This must be done manually after successfully disabling feedback.

    Example:

    Copy image Copy Code
    PS C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100> Disable-RemoteMonitoring

If you see any errors or information that leads you to believe that the command was not successful contact CSS for assistance.

There is no harm in running the Disable-RemoteMonitoring command multiple times.

See Also

Concepts

Monitor the Appliance by Using the Admin Console (Analytics Platform System)
Monitor the Appliance by Using System Views (Analytics Platform System)
Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)
Use a DNS Forwarder to Resolve Non-Appliance DNS Names (Analytics Platform System)


Software Servicing (Analytics Platform System)

[image]

Software Servicing (Analytics Platform System)
See Also Send Feedback

This section summarizes the software servicing requirements for Analytics Platform System appliances, including WSUS and Analytics Platform System hotfixes.

Software Servicing Basics

WSUS: Your Analytics Platform System appliance needs to be configured to receive updates from Windows Server Update Services (WSUS). These updates include important changes to appliance software. After they are configured, many updates will install automatically and do not require hands-on management. Typically, WSUS updates are configured during the Configure Windows Server Update Services (WSUS) (Analytics Platform System) step performed during new appliance setup. If not, this configuration step can be performed later. For information on WSUS, see the WSUS website Guide.

Hotfixes: Additionally, you may need to apply Analytics Platform System hotfixes. A hotfix is a software update that is created for a specific customer to resolve an issue with the Analytics Platform System software. Each hotfix is an executable file that installs the fix for the customer-specific issue. Each hotfix also contains an accumulation of all the previously released software updates for Windows, SQL Server, and Analytics Platform System. If you need to install a hotfix, Microsoft support will provide you with the hotfix and instructions.

Scope of Updates: Applying a hotfix or service pack to the Analytics Platform System must take the entire appliance offline. That is, both the PDW and HDInsight regions will be affected.

SSIS Destination Adapter and Client Tools: When applying a hotfix that includes changes to the SSIS Destination Adapter MSI's or Client Tools MSI, the MSI files will be updated in the C:\PDWINST\ClientTools directory on the Control node. The hotfix does not automatically install the components from the updated MSI files. To update those components, the customer must uninstall the old versions of the components, and install the new versions from the updated MSI files. When uninstalling a hotfix that includes changes to the SSIS Destination Adapter MSI's or Client Tools MSI, the MSI files for those components will be reverted to the previous versions. To revert those components to a previous version, the customer must uninstall the existing (newer) versions of the components, and reinstall the older versions from the reverted MSI files.

Software Servicing Topics

The following topics describe how to manage software servicing on the appliance:

  • Download and Apply Microsoft Updates (Analytics Platform System)

  • Uninstall Microsoft Updates (Analytics Platform System)

  • Apply Analytics Platform System Hotfixes (Analytics Platform System)

  • Uninstall Analytics Platform System Hotfixes (Analytics Platform System)

See Also


Download and Apply Microsoft Updates (Analytics Platform System)

[image]

Download and Apply Microsoft Updates (Analytics Platform System)
See Also Send Feedback

This topic discusses how to download updates from the Microsoft Update Catalog to Windows Server Update Services (WSUS) and apply those updates to the Analytics Platform System appliance servers. Microsoft Update will install all applicable updates for Windows and SQL Server. WSUS is installed on the VMM virtual machine of the appliance.

Before You Begin

Caution note Caution

Do not attempt to apply updates if your appliance or any appliance component is down or in a failed over state. In that case, contact support for assistance.

Do not apply Microsoft Updates while the appliance is in use. Applying updates can cause appliance nodes to reboot. The updates should be applied during a maintenance window when the appliance is not being used.

Prerequisites

Before performing these steps, you need to:

  • Configure WSUS on your appliance by following the instructions in Configure Windows Server Update Services (WSUS) (Analytics Platform System).

  • Knowledge of a Fabric Domain Administrator account login information.

  • Have a login with permissions to access the Analytics Platform System Admin Console and view appliance state information.

  • In most cases, WSUS needs to access servers outside of the appliance. To support this usage scenario the Analytics Platform System DNS can be configured to support an external name forwarder that will allow the Analytics Platform System hosts and Virtual Machines (VMs) to use external DNS servers to resolve names outside of the appliance. For more information, see Use a DNS Forwarder to Resolve Non-Appliance DNS Names (Analytics Platform System).

To download and apply Microsoft updates

Verify the appliance state indicators

  1. Open the Admin Console and navigate to the Appliance State page. For more information, see Monitor the Appliance by Using the Admin Console (Analytics Platform System)

  2. Verify the status indicators for all nodes on the Appliance State.

    • It is safe to continue with green or NA indicators.

    • Evaluate non-critical (yellow) warning errors. In some cases warning messages will not block updates. If there is a non-critical disk volume error that is not on the C:\ drive, you can proceed to the next step before resolving the disk volume error.

    • Most red indicators must be resolved before continuing. If there are disk failures, use the Admin Console Alerts page to verify there is no more than one disk failure within each server or SAN array. If there is no more than one disk failure within each server or SAN array, you can proceed to the next step before fixing the disk failures. Be sure to contact Microsoft support to fix the disk failures as soon as possible.

Synchronize the WSUS server

  1. Log on to the VMM virtual machine as a domain administrator.

  2. In the Server Manager Dashboard, on the Tools menu, click Windows Server Update Services (wsus.msc).

  3. In the WSUS Admin Console, click on Synchronizations.

  4. If synchronization is not running, click Synchronize Now in the right pane. In the bottom pane, there will be a synchronization status. Wait until the synchronization has completed.

Approve Microsoft updates in WSUS

  1. In the left pane, of the WSUS console, click All Updates.

  2. In the All Updates pane, click the Approval drop-down menu, set Approval to Any Except Declined. Click the Status drop-down menu, set Status to Any. Click Refresh.

    Right-click the Title column and select File Status to verify the file status after the download completes.

    You can also select Critical Updates or Security Updates in the left pane and view available updates for these categories.

    Select all updates and change status to Any.

  3. Select all updates, and then click the Approve link in the right pane.

    You can also right-click the selected updates, and then click Approve. You may be prompted to accept the “Microsoft Software License Terms”. If so, click I Accept in the window to continue.

    Select all updates that apply and click Approve.

  4. Select the appliance server group you created in Configure Windows Server Update Services (WSUS) (Analytics Platform System).

  5. Click Approved for Install, and then click OK.

    Approve updates for your computer group.

  6. In the Approval Progress dialog box, when the approval process is complete, click Close.

    Close window when updates are approved.

Verify that the updates are in WSUS

  • Verify the file status of all updates. Each file needs to have a green arrow icon to the left of the title. This indicates the file is ready for installation.

    File status is successful

    Before installing the updates, make sure that they are all downloaded and available in the WSUS console.

To verify that all updates are downloaded

  • Check the Download Status of updates in the WSUS console as shown in the following screenshot. Check that Updates needing files is 0 to confirm that all updates are downloaded. If this number is more than zero, you may need to go back and approve additional updates.

    Verify that all updates are downloaded.

Apply Microsoft updates

  1. Before you start, open the Monitor the Appliance by Using the Admin Console (Analytics Platform System), click the Appliance State tab, and verify that the Cluster and Network columns show green (or NA) for all nodes. If any alerts exist in either of these columns, the appliance might not be able to install updates properly. Address all existing alerts in the Cluster and Network columns before proceeding.

  2. Log on to the <domain_name>-HST01 node as the Fabric Domain Administrator.

  3. To apply all updates approved for WSUS, run the Update program. See Run the Update Program below for instructions.

Verify the updates on all nodes

  1. From the VMM node, launch the WSUS Admin Console. This application can be found under Start, Administrative Tools, Windows Server Update Services.

  2. Expand Computers.

  3. Expand All Computers.

  4. Select the appliance server group you created in Configure Windows Server Update Services (WSUS) (Analytics Platform System).

  5. In the Status drop-down menu, select Any and click Refresh.

  6. Expand Update Services, <appliance name>-VMM, Updates, All Updates, where <appliance name> is your appliance name.

  7. In the All Updates window set Approval to Any Except Declined.

  8. In the All Updates window, set Status to Failed or Needed.

  9. Click Refresh.

  10. If Updates needed is greater than zero, contact support for assistance.

Ensure there are no critical alerts in the SQL Server PDW Admin Console

  1. Open the Admin Console, click the Appliance State tab. See Monitor the Appliance by Using the Admin Console (Analytics Platform System).

  2. Verify that the Cluster and Network columns show green (or NA) for all nodes. If any alerts exist in either of these columns, the appliance might not be able to install updates properly. Contact support if there are any critical alerts.

[TOP]

Run the Update Program

Follow these instructions to run the Analytics Platform System Update program.

Note Note

The WSUS system is designed to run asynchronously may take some time to fully apply all updates. Initiating an update schedules an update but does not guarantee immediate update activity.

  1. Make sure you are logged into the HST01 node as the Fabric Domain Administrator.

  2. Open a Command Prompt window and enter the following commands. Replace <parameter> with the designated information.

To Run the Microsoft Update:

Copy image Copy Code
C:\pdwinst\media\setup.exe /action="MicrosoftUpdate" /DomainAdminPassword="<password>"

To report the Microsoft Update status:

Copy image Copy Code
C:\pdwinst\media\setup.exe /action="ReportMicrosoftUpdateClientStatus" /DomainAdminPassword="<password>"

See Also

Concepts

Uninstall Microsoft Updates (Analytics Platform System)
Apply Analytics Platform System Hotfixes (Analytics Platform System)
Uninstall Analytics Platform System Hotfixes (Analytics Platform System)
Software Servicing (Analytics Platform System)


Uninstall Microsoft Updates (Analytics Platform System)

[image]

Uninstall Microsoft Updates (Analytics Platform System)
See Also Send Feedback

This topic describes how to uninstall a previously installed Microsoft update on the Analytics Platform System appliance.

Before You Begin

Prerequisites

To perform these steps, you will need:

  • A Analytics Platform System login with permissions to access the Admin Console to monitor the appliance.

  • Knowledge of the Fabric Domain Administrator account to login to the <Fabric Domain>-HST01 node.

To uninstall Microsoft updates

  1. Login to the <Fabric Domain>-HST01 node as the Fabric Domain Administrator.

  2. To uninstall all updates that are approved for WSUS to uninstall, open a Command Prompt window and enter the following command. Replace the placeholder items < > with the appropriate information.

    Copy image Copy Code
    C:\pdwinst\media\setup.exe /action="RemoveMicrosoftUpdate" /DomainAdminPasswords="<password>"

See Also

Concepts

Download and Apply Microsoft Updates (Analytics Platform System)
Apply Analytics Platform System Hotfixes (Analytics Platform System)
Uninstall Analytics Platform System Hotfixes (Analytics Platform System)
Software Servicing (Analytics Platform System)


Apply Analytics Platform System Hotfixes (Analytics Platform System)

[image]

Apply Analytics Platform System Hotfixes (Analytics Platform System)
See Also Send Feedback

This topic discusses how to apply hotfixes to the Analytics Platform System software.

Before You Begin

Caution note Caution

Do not attempt to apply a Analytics Platform System hotfix if your appliance or any appliance component is down or in a failed over state. In that case, contact support for assistance.

Caution note Caution

Do not apply a Analytics Platform System hotfix while the appliance is in use. Applying a hotfix can cause appliance nodes to reboot. The hotfix should be applied during a maintenance window when the appliance is not being used.

Prerequisites

To perform these steps, you will need:

  • An Analytics Platform System login with permissions to access the Admin Console to monitor the appliance state. See Grant Permissions to Use the Admin Console (SQL Server PDW).

  • Knowledge of the Fabric Domain Administrator account to connect to the <domain_name>-HST01 node.

To apply a Analytics Platform System hotfix

Unlike the Microsoft updates, the hotfixes for the Analytics Platform System software are not handled through WSUS. They have a different workflow and are installed by running a hotfix package.

  1. Verify appliance state indicators.

    1. Open the Admin Console and navigate to the Appliance State page. For more information, see Monitor the Appliance by Using the Admin Console (Analytics Platform System)

    2. All red or yellow indicators must be resolved before you proceed to the next step. A couple exceptions to this are:

      • If there are disk failures, use the Admin Console Alerts page to verify there is no more than one disk failure within each server or SAN array. If there is no more than one disk failure within each server or SAN array, you can proceed to the next step before fixing the disk failure(s). Be sure to contact Microsoft support to fix the disk failure(s) as soon as possible.

      • If there is a non-critical (yellow) disk volume error that is not on the C:\ drive, you can proceed to the next step before resolving the disk volume error.

  2. Install the Analytics Platform System hotfix

    1. Login to the <appliance_domain>-HST01 node as the domain administrator.

    2. Use the Run as administrator option to open a Command Prompt.

    3. Run the following command, replacing <HotfixPackageName> with the name of the hotfix executable package, and replacing the other placeholder items < > with the appropriate information.

      Copy image Copy Code
      <HotfixPackageName> /DomainAdminPassword="<password>"

    4. Follow the steps as presented by the hotfix package.

[TOP]

See Also

Concepts

Download and Apply Microsoft Updates (Analytics Platform System)
Uninstall Microsoft Updates (Analytics Platform System)
Uninstall Analytics Platform System Hotfixes (Analytics Platform System)
Software Servicing (Analytics Platform System)


Uninstall Analytics Platform System Hotfixes (Analytics Platform System)

[image]

Uninstall Analytics Platform System Hotfixes (Analytics Platform System)
See Also Send Feedback

The following steps describe how to uninstall a previously installed Analytics Platform System hotfix.

Before You Begin

Prerequisites

To perform these steps, you will need:

  • An Analytics Platform System login with permissions to access the Admin Console to monitor the appliance.

  • The Domain Administrator account to login to the <appliance_domain>-HST01 node.

  • The Knowledge Base article number for the hotfix to uninstall.

To uninstall a SQL Server PDW hotfix

  1. Log on to the <appliance_domain>-HST01 node as the Fabric Domain Administrator.

  2. Use the Run as Administrator option to open a Command Prompt.

  3. Change directories to C:\PDWINST\Patches\<kbarticle>\media where <kbarticle> is the Knowledge Base article number for the hotfix to uninstall.

    Copy image Copy Code
    cd /d c:\PDWINST\Patches\<kbarticle>\media

  4. To remove the hotfix, run the following command.

    Copy image Copy Code
    setup.exe /action="removepatch" /DomainAdminPassword="<password>"

See Also

Concepts

Download and Apply Microsoft Updates (Analytics Platform System)
Uninstall Microsoft Updates (Analytics Platform System)
Apply Analytics Platform System Hotfixes (Analytics Platform System)
Software Servicing (Analytics Platform System)


Power the APS Appliance On or Off (Analytics Platform System)

[image]

Power the APS Appliance On or Off (Analytics Platform System)
See Also Send Feedback

This topic describes how to power on or power off your Analytics Platform System appliance that is running Parallel Data Warehouse, and optionally running an HDInsight region. Use this topic when a Analytics Platform System appliance is moved, or to power on an appliance after a catastrophic power failure.

Powering the appliance on and off is not the same as starting and stopping the appliance services. For information on that subject, see PDW Services Status (Analytics Platform System). For information about powering on or off a SQL Server 2008 Parallel Data Warehouse, see the SQL Server 2008 Parallel Data Warehouse help file. For information about powering on or off a SQL Server 2012 AU1 or AU2 Parallel Data Warehouse, see the help file for those versions.

When these instructions specify connecting to a SQL Server PDW node, the connection can be local using attached devices (KVM) or remote using Remote Desktop. Some actions must be physical (turning on a power switch), and some (such as shut down) can be physical or by using Windows commands.

Connections to SQL Server PDW nodes can be made using the IP addresses assigned to the nodes, or from the HST01 computer by using either the Failover Cluster Manager (cluadmin.msc) or Hyper-V Manager (virtmgmt.msc) applications and right-clicking the node name.

Contents

  • Power Off the Appliance

  • Power On the Appliance

Power Off the Appliance

Before you begin

Before powering off the appliance, you should end all activity on the appliance. To end all activity:

  • Use the Sessions page of the Admin Console to identify the current users. Contact them and ask them to log off.

  • If necessary you can use the KILL statement to force the termination of a client connection. Use caution when killing connections. When interrupted, some transactional processes, such as a long running update, must rollback activity before SQL Server can complete the recovery of the database. Rolling back a partially completed update or delete, can be time consuming.

To power off the appliance

Caution note Caution

All steps must be performed in the exact order listed and each step must complete before the next step is performed, unless otherwise noted. Performing steps out of order or without waiting for each step to complete can result in errors when the appliance is powered up at a later time.

  1. Connect to the PDW Control node (PDW_region-CTL01 ) and login with the Analytics Platform System appliance domain administrator account.

  2. Run C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100\dwconfig.exe to open the Configuration Manager.

  3. In the Configuration Manager, under the Parallel Data Warehouse Topology menu, click the Services Status tab, and click Stop Region to stop PDW services.

  4. If there is an HDInsight region, under the HDInsight Topology menu, click the Services Status tab, and click Stop Region to stop HDInsight services.

  5. Connect to appliance_domain-HST01 and login with the appliance domain administrator account.

  6. Using the Failover Cluster Manager connect to the appliance_domain-WFOHST01 cluster, if not automatically connected, and then in the Navigation pane, click Roles. In the Roles pane:

    1. Multi-select all of the virtual machines. Right-click them, and select Shut Down.

    2. Wait for all selected VMs to finish shutting down.

  7. If there is an HDInsight region:

    1. Connect to the HDInsight cluster. To do this, right-click on Failover Cluster Manager, select Connect to Cluster, and specify appliance_domain-WFOHST02 for the cluster name.

    2. Under the HDInsight cluster, click Roles. In the Roles pane:

      1. Multi-select all of the virtual machines, right-click them, and select Shut Down.

      2. Wait for the virtual machines to shut down.

  8. Close the Failover Cluster Manager application.

  9. Shut down all the servers except appliance_domain-HST01.

  10. Shut down the appliance_domain-HST01 server.

  11. Shut down the Power Distribution Units (PDUs).

[TOP]

Power On the Appliance

To power on the appliance

Caution note Caution

All steps must be performed in the exact order listed and each step must complete before the next step is performed, unless otherwise noted. Performing steps out of order or without waiting for each step to complete can result in startup errors.

  1. Power on the Power Distribution Units (PDU's), and wait for the switches to automatically start.

  2. Power on the appliance_domain-HST01 server.

  3. Log in to appliance_domain-HST01 as the appliance domain administrator.

  4. Start the Hyper-V Manager program (virtmgmt.msc) and connect to appliance_domain-HST01 if not connected by default.

    1. If you cannot connect by name because the PDW_region-AD01 is not running, try connecting by using the IP address.

    2. In the Virtual Machines pane, locate PDW_region-AD01 and confirm that it is running. If not, start this VM and wait for it to be fully started.

  5. Power on the rest of the servers in the appliance.

  6. While on HST01 logged on as the appliance domain administrator, from Hyper-V Manager:

    1. Connect to appliance_domain-HST02.

    2. In the Virtual Machines pane, locate PDW_region-AD02 and confirm that it is running. If not, start this VM and wait for it to be fully started.

  7. Using the Failover Cluster Manager connect to the appliance_domain-WFOHST01 cluster, if not automatically connected, and then in the Navigation pane, click Roles. In the Roles pane:

    1. Multi-select all of the virtual machines, right-click them, and then click Start.

    2. Wait for all selected VMs to finish starting before proceeding to the next step.

    3. If necessary for the VMs that failed over, shut them down, move them, and restart them on the proper primary host.

  8. If the appliance has an HDInsight region, connect to the HDInsight cluster. (To do this, right-click on Failover Cluster Manager, select Connect to Cluster, and specify appliance_domain-WFOHST01 for the cluster name.)

    1. Under the HDInsight cluster, click Roles. In the Roles pane.

      1. Multi-select all of the virtual machines, right-click them, and select Start,

      2. Wait for all selected VMs to finish starting before proceeding to the next step.

      3. If necessary for the VMs that failed over, shut them down, move them and restart them on the proper primary host.

  9. Disconnect from HST01 if you wish.

  10. Connect to PDW_region-CTL01 using the appliance domain administrator account.

  11. Run C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100\dwconfig.exe to launch the Configuration Manager.

  12. In the Configuration Manager, in the Parallel Data Warehouse Topology menu, click the Services Status tab, and click Start Region to start PDW services.

  13. If the appliance has an HDInsight region, in the HDInsight Topology menu, click the Services Status tab, and click Start Region to start the HDInsight services.

To verify the appliance health

After the appliance has started, open the Admin Console and check the Health page for alerts that might indicate failure conditions. For more information, see Monitor the Appliance by Using the Admin Console (Analytics Platform System).

[TOP]

See Also

Concepts

Appliance Management Tasks (Analytics Platform System)


Appliance Monitoring (Analytics Platform System)

[image]

Appliance Monitoring (Analytics Platform System)
See Also Send Feedback

This Appliance Monitoring Guide describes the tools and tasks for monitoring the SQL Server PDW appliance.

Contents

  • Monitoring Basics and Tools

  • Related Monitoring Tasks

Monitoring Basics and Tools

The values and information that can be monitored on the SQL Server PDW appliance are extensive. For example, the following are typical monitoring tasks.

  • Check for any alert issued by SQL Server PDW.

  • Monitor for failed hardware.

  • Monitor for network connectivity problems.

  • Check for errors returned to users during query processing.

  • View the number of currently active sessions and queries.

  • Check the status of loads, backups, and restores.

Appliance Monitoring Tools

There are multiple tools available to monitor the appliance.

Admin Console

SQL Server PDW has an Admin Console. This is a web-based tool that displays information about queries, loads, backup and restore, locks, sessions, alerts, and appliance state. The Admin Console runs on the appliance; users connect to the Admin Console through Internet Explorer. For more information, see:

  • Monitor the Appliance by Using the Admin Console (Analytics Platform System)

PDW Admin Console Alerts
System Views

SQL Server PDW includes comprehensive system views that enable you to obtain detailed information about the appliance health, state, and performance. For a list of system views for monitoring tasks, see:

  • Monitor the Appliance by Using System Views (Analytics Platform System)

System Center Operations Manager (SCOM)

SQL Server PDW has extensive integration with Systems Center Operations Manager. The management packs for SQL Server PDW are available as a free download. For more information about using System Center to monitor SQL Server PDW, see the following:

  • Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)

Custom Solutions

For situations when System Center is not available with your data center monitoring tools, you can monitor the appliance by using a third-party monitoring solution. Most monitoring solutions support Transact-SQL integration, so the system administrator can implement direct Transact-SQL queries against your PDW appliance.

If your monitoring solution does not support direct Transact-SQL queries, or you do not have a monitoring tool, then you can use scripts to perform monitoring tasks, such as sending email when an alert occurs. The TechNet wiki contains a scripted monitoring solution example.

  • Power Shell Monitoring Example for SQL Server PDW

If you can’t access this article, see Join Our Communities (Analytics Platform System).

Related Monitoring Tasks

Monitoring Task | Description
Monitor the appliance by using the Admin Console. | Monitor the Appliance by Using the Admin Console (Analytics Platform System)
Monitor the appliance by using System Views. | Monitor the Appliance by Using System Views (Analytics Platform System)
Monitor the appliance by using System Center | Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)
Monitor the state of the appliance. | Monitor Appliance Health State (Analytics Platform System)
Heartbeat Monitoring. | Send Telemetry Feedback to Microsoft (SQL Server PDW)
Track appliance alerts. | Track Appliance Alerts (Analytics Platform System)
Determine how much capacity is being used. | View Capacity Utilization (Analytics Platform System)
Determine how often to poll the appliance. | Determine Polling Frequency (Analytics Platform System)
When a cluster failure occurs, determine which cluster node failed. | Determine Which Cluster Node Failed (Analytics Platform System)
Monitor loads. | Monitor Loads (SQL Server PDW)
Monitor backups and restores. | Monitor Backups and Restores (SQL Server PDW)
Monitor the active queries. | Monitoring Active Queries (SQL Server PDW)

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Appliance Management Tasks (Analytics Platform System)


Monitor the Appliance by Using the Admin Console (Analytics Platform System)

[image]

Monitor the Appliance by Using the Admin Console (Analytics Platform System)
See Also Send Feedback

The Admin Console is a SQL Server PDW web application that surfaces the appliance state, health, and performance information. Users connect to the Admin Console through Internet Explorer.

Contents

  • About the Admin Console

  • Connect to the Admin Console

  • Admin Console Tasks

About the Admin Console

Appliance Console Home
Appliance
Home

Provides a quick summary of the appliance state.

Health

Displays the appliance topology with indicators showing the health of each monitored component within each node. Allows you to view the current status of individual nodes and properties of the node components.

Displays hardware and software alerts.

Performance Monitor

Displays performance monitor graphs.

Parallel Data Warehouse
Home

Provides a quick summary of the PDW state.

Sessions

Displays active PDW user sessions. This can help for monitoring resource contention.

Queries

Displays a list of running queries and recently completed queries. It displays related errors, if any. Also provides the ability to view details of the query execution plan and node execution information.

Loads

Displays load plans, the current state of PDW loads, and related errors, if any.

Backups/Restores

Displays a log of PDW backup and restore operations.

Health

Displays the PDW topology with indicators showing the health of each monitored component within each node. Allows you to view the current status of individual nodes and properties of the node components.

Displays hardware and software alerts.

Resources

Displays a list of PDW resource locks and their current status.

Storage

Summarizes the PDW storage utilization.

Performance Monitor

Displays PDW performance monitor graphs.

HDInsight
Home

Provides a quick summary of the HDInsight state.

HDFS

Summarizes the HDInsight space utilization and lists the top 10 space consumers.

Map/Reduce

Summarizes the status of MapReduce jobs.

Health

Displays the HDInsight topology with indicators showing the health of each monitored component within each node. Allows you to view the current status of individual nodes and properties of the node components.

Displays hardware and software alerts.

Storage

Summarizes the HDInsight storage utilization.

Performance Monitor

Displays performance monitor graphs.

Note Note

The admin console has a 1024x768 screen resolution. The admin console displays best with a screen resolution of 1280 X 1024 or higher.

[TOP]

Connect to the Admin Console

To connect to the Admin Console, requires:

  • At least Internet Explorer version 10.

  • Permissions to access the Admin Console. See Grant Permissions to Use the Admin Console (SQL Server PDW).

  • The IP address of the Control node cluster. Obtain this from your SQL Server PDW administrator.

To connect to the Admin Console, use Internet Explorer and https to browse to the IP address of the Control node cluster. For example, if the IP address of the Control node cluster is 10.192.63.102, enter https://10.192.63.102 in your browser address bar. The first screen will request your LOGIN and PASSWORD. Provide either a SQL Server Authentication login and password, or a Windows Authentication login and Windows password. If using a Windows Authentication login, the Admin Console will use impersonation.

[TOP]

Admin Console Tasks

The Admin Console provides the ability to monitor the following:

Information Type | How to Access in the Admin Console
Overall status of the appliance | Click Appliance State in the top menu, or Home.
Alerts | Click Alerts. For more information, see Understanding Admin Console Alerts (Analytics Platform System).
Appliance components and their status | Click Appliance State in the top menu, or Home.
Monitor requests (including queries, loads, backups, and restores) | Click Sessions to see currently active or recent sessions. Click Queries to see currently active or recent queries. The information displayed for queries includes loads, backups, and restores. Click Locks to see active locks.
Monitor additional information for loads, backups, and restores. | Click Loads or Backups/Restores.
Performance information | Click Performance Monitor.

[TOP]

See Also

Concepts

Appliance Monitoring (Analytics Platform System)


Monitor the Appliance by Using System Views (Analytics Platform System)

[image]

Monitor the Appliance by Using System Views (Analytics Platform System)
See Also Send Feedback

This topic lists the system views that you can use for monitoring SQL Server PDW.

To Monitor the Appliance by Using System Views

SQL Server PDW includes comprehensive system views that enable you to obtain detailed information about the appliance health, state, and performance. This table provides links to system views that can be used for each monitoring capability.

PDW system views alerts
Information Type | Related System Views
Overall status of the appliance | sys.dm_pdw_sys_info (SQL Server PDW)
Alerts | sys.pdw_health_alerts (SQL Server PDW) sys.dm_pdw_component_health_alerts (SQL Server PDW) sys.dm_pdw_component_health_active_alerts (SQL Server PDW) sys.dm_pdw_component_health_status (SQL Server PDW)
Appliance components and their status | sys.pdw_health_component_groups (SQL Server PDW) sys.pdw_health_components (SQL Server PDW) sys.pdw_health_component_properties (SQL Server PDW) sys.pdw_health_component_status_mappings (SQL Server PDW) sys.dm_pdw_nodes (SQL Server PDW)
Monitor requests (including queries, loads, backups, and restores) | sys.dm_pdw_exec_sessions (SQL Server PDW) sys.dm_pdw_exec_requests (SQL Server PDW) sys.dm_pdw_request_steps (SQL Server PDW) sys.dm_pdw_sql_requests (SQL Server PDW) sys.dm_pdw_dms_workers (SQL Server PDW) sys.dm_pdw_waits (SQL Server PDW) sys.dm_pdw_errors (SQL Server PDW) sys.pdw_distributions (SQL Server PDW)
Monitor additional information for loads, backups, and restores. | sys.pdw_loader_backup_runs (SQL Server PDW) sys.pdw_loader_backup_run_details (SQL Server PDW) sys.pdw_loader_run_stages (SQL Server PDW)
OS-level logs and performance information | sys.dm_pdw_os_performance_counters (SQL Server PDW) sys.dm_pdw_os_event_logs (SQL Server PDW) sys.dm_pdw_os_threads (SQL Server PDW)

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Appliance Monitoring (Analytics Platform System)


Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)

[image]

Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)
See Also Send Feedback

This describes how to use System Center Operations Manager to monitor SQL Server PDW and HDInsight.

Parent Topic: Appliance Management Tasks (Analytics Platform System)

Before You Begin

Prerequisites

  1. System Center Operations Manager 2007 R2, 2012, or 2012 SP1 must be installed and running.

  2. The management packs to monitor SQL Server PDW and HDInsight must be installed, imported, and configured. Use the following for instructions to perform these tasks.

    • Install the SCOM Management Packs (Analytics Platform System)

    • Import the SCOM Management Pack for PDW (Analytics Platform System)

    • Import the SCOM Management Pack for HDInsight (Analytics Platform System)

    • Configure SCOM to Monitor Analytics Platform System (Analytics Platform System)

To Monitor SQL Server PDW with SCOM

After configuring the SCOM Management Packs, click on the Monitoring Pane of SCOM and drill down to SQL Server Appliance and then Microsoft SQL Server Parallel Data Warehouse. Underneath Microsoft SQL Server Parallel Data Warehouse, there are four choices: Alerts, Appliances, Appliance Diagram, and nodes.

Alerts

Alerts are where you can find the current alerts to manage.

Appliances

Appliances are where you will find the currently discovered and monitored SQL Server PDW Appliances in your environment. If an appliance does not show up here and you have created the ODBC connection for it, then there may be something wrong with your PDWWatcher account. If they show up as “Not monitored” there may be something wrong with your PDWMonitor account. Please be patient as SCOM does not make changes in real time, but periodically checks for new appliances to monitor and periodically sends queries to appliances for monitoring.

Appliances Diagram

The Appliances Diagram Page is where you can get a look at the health of your appliance with a tree view:

Nodes

Finally, the Nodes view allows you to see the health of your appliance through each node:

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Understanding Admin Console Alerts (Analytics Platform System)


Install the SCOM Management Packs (Analytics Platform System)

[image]

Install the SCOM Management Packs (Analytics Platform System)
See Also Send Feedback

Follow these steps to download and install the System Center Operations Manager (SCOM) Management Packs for SQL Server PDW. The Management Packs are required to monitor SQL Server PDW from SCOM.

  • Before You Begin

  • Step 1: Download the Management Packs

  • Step 2: Install the Management Packs

Parent Topic: Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)

Before You Begin

Prerequisites

System Center Operations Manager must be installed and running. SQL Server PDW 2012 requires System Center Operations Manager 2007 R2, System Center Operations Manager 2012, or System Center Operations Manager 2012 service pack 1.

Step 1: Download the Management Packs

For the APS PDW workload, download the System Center Management Pack for the Microsoft Analytics Platform System.

For the appliance management, download the SQL Server Appliance Base Management Pack.

For older versions of PDW without APS, download the System Center Monitoring Pack for Microsoft SQL Server 2012 Parallel Data Warehouse Appliance.

For the HDInsight workload, download the System Center Management Pack for HDInsight.

Step 2: Install the Management Packs

Install the SQL Server Appliance Base Management Pack

  1. To run the install, double-click on the downloaded SQL Server Appliance Base Management Pack.

  2. Accept the License Agreement, and click Next.

    Accept the License Agreement

  3. Select your own installation folder, or use the default Management Pack Installation Folder.

    Select Installation Folder

  4. Click Install.

  5. Click Close.

Install the Monitoring Pack for SQL Server PDW Appliance

  1. To run the install, double-click on the downloaded SQL Server PDW Appliance Management Pack.

  2. Accept the License Agreement, and click Next.

    Accept the license agreeement

  3. Choose the directory that will hold the extracted files. The default Management Pack installation folder is shown by default. Select the default, or select your own installation folder.

    Select installation folder

  4. Click Install.

  5. Click Close.

Next Step

Now that you have the Management Packs installed, continue to the next step: Import the SCOM Management Pack for PDW (Analytics Platform System).

See Also


Import the SCOM Management Pack for PDW (Analytics Platform System)

[image]

Import the SCOM Management Pack for PDW (Analytics Platform System)
See Also Send Feedback

Follow these steps to import the System Center Operations Manager (SCOM) Management Packs for SQL Server PDW. The Management Packs are required to monitor SQL Server PDW from SCOM.

  • Before You Begin

  • Step 1: Import the SQL Server Appliance Base Management Pack

  • Step 2: Import the SQL Server PDW Appliance Management Pack

Parent Topic: Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)

Before You Begin

Prerequisites

System Center Operations Manager 2007 R2 must be installed and running.

The management packs must be installed. See Install the SCOM Management Packs (Analytics Platform System).

Step 1: Import the SQL Server Appliance Base Management Pack

  1. Log on to the computer with an account that is a member of the Operations Manager Administrators role for the Operations Manager 2007 management group.

  2. In the Operations console, click Administration.

  3. Right-click the Management Packs node, and then click Import Management Packs.

    Click Import Management Packs

  4. In the list of management packs, select the management pack that you want to import, click Select, and then click Add.

    Management pack list

  5. Search for Appliance and then drill down into SQL Server Appliance Base and then click Add the two choices.

    SQL Server appliance base

  6. Once the two Management Packs were in the bottom selected pane, then click OK.

    Select both management packs

  7. Click Install.

  8. Once Complete, click Close.

    Once complete, click Close

Import the Monitoring Pack for Microsoft SQL Server 2008 R2 Parallel Data Warehouse Appliance

  1. Right-click the Management Packs node, and then click Import Management Packs.

  2. Choose Add from disk….

    Right-click Management Packs

  3. Go to the location where you extracted the SQL Server PDW Management Packs and choose the three management packs that are in the "Selected Management packs to import" section. You can do this by selecting the first one, clicking Shift, and selecting the last one. Once they are all selected, click Open.

    Select management packs

  4. Click Install.

  5. Click Close.

Next Step

Now that you have imported the Management Packs, continue to the next step: Configure SCOM to Monitor Analytics Platform System (Analytics Platform System).

See Also


Import the SCOM Management Pack for HDInsight (Analytics Platform System)

[image]

Import the SCOM Management Pack for HDInsight (Analytics Platform System)
See Also Send Feedback

Follow these steps to import the System Center Operations Manager (SCOM) Management Packs for Analytics Platform System HDInsight. The Management Packs are required to monitor HDInsight from SCOM.

Import the HDInsight Management Pack for System Center 2012

  1. Install the HDI Management Pack as described in Install the SCOM Management Packs (Analytics Platform System).

  2. In System Center 2012, right-click the Management Packs node, and then click Import Management Packs.

  3. Point to Add, and then click Add from disk.

    HDI SCOM Pack install page 1

  4. Navigate to the folder where you extracted the HDInsight Management Packs for System Center 2012 and select the three management packs in the Selected Management packs to import section, and then click Open.

    HDI SCOM Pack install page 2

  5. In the Select Management Packs dialog box, with the three HDInsight Management Packs selected, click Install.

    HDI SCOM Pack install page 3

  6. When the status for each pack is Imported, click Close.

    HDI SCOM Pack install page 4

Configure the Account used by the Management Pack to Connect to HDInsight

The HDInsight Management Pack interacts with the cluster using the Ambari API. To configure the discovery wizard correctly, an account is needed for the connection to RunAs.

  1. In System Center 2012, in the Administration section, expand Run As configuration, expand to Accounts, and then right-click Run As Account, to start the Create Run As Account Wizard.

  2. On the General Properties page, in the Run As account type box, click Basic Authentication, and then click Next. (HDInisght only supports Basic Authentication.)

    HDI SCOM Pack install page 5

  3. On the Credentials page, enter an account name and password, and then click Next.

    Security note Security Note

    You should use a HDInsight Cluster Admin account created using Configuration Manager, not a high privileged account such as a HDInsight domain administrator. For more information, see HDInsight User Management (Analytics Platform System).

    HDI SCOM Pack install page 6

  4. On the Distribution Security page, typically select Less secure, and then click Create.

    Security note Security Note

    Less secure is a reasonable option because the APS has limited outside connectivity. See your domain administrator for information about your company policy.

    HDI SCOM Pack install page 7

See Also

Concepts

Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)
HDInsight Management Tasks Using SCOM (Analytics Platform System)


Configure SCOM to Monitor Analytics Platform System (Analytics Platform System)

[image]

Configure SCOM to Monitor Analytics Platform System (Analytics Platform System)
See Also Send Feedback

Follow these steps to configure the System Center Operations Manager (SCOM) Management Packs for Analytics Platform System. The Management Packs are required to monitor Analytics Platform System from SCOM.

  • Before You Begin

  • Step 1: Configure System Data Source Name (DSN)

  • Step 2: Create Run-As Profile in System Center for PDW Watcher

  • Step 3: Create Run-As Profile in System Center for PDW Monitor

Parent Topic: Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)

Before You Begin

Prerequisites

System Center Operations Manager 2007 R2 must be installed and running.

The management packs must be installed and configured. See Install the SCOM Management Packs (Analytics Platform System) and Import the SCOM Management Pack for PDW (Analytics Platform System).

Step 1: Configure System Data Source Name (DSN)

For discovery and monitoring of the Analytics Platform System, the SQL Server PDW ODBC DSN must be configured on the agent (watcher node) computer.

To configure the client, do the following:

  1. Open the ODBC Data Source Administrator window.

  2. On the System DSN tab, click Add.

  3. Select Microsoft SQL Server 2012 Parallel Data Warehouse Appliance ODBC Client.

  4. Click Finish.

  5. Enter the:

    • Data Source Name. This name is used as Appliance Name in the SCOM console.

    • SQL Server PDW Host. This is the network address used to connect to PDW instance.

    • SQL Server PDW Port. The default port is 17000.

    • master as Database.

  6. Click OK.

Step 2: Create Run-As Profile in System Center for PDW Watcher

The PDW Watcher Account Discovers new PDW Appliances to monitor by watching for new PDW ODBC connections on the local SCOM Server.

  1. In the Operations Console, Click on Administration Pane.

  2. In the Administration pane, right click on Accounts under Run As Configuration and choose Create Run As Account…

    Create Run As Account

  3. Click Next on the Introduction Page.

  4. Type PDWWatcher in the Display name section. Click Next.

    Display name section

  5. Click New…

  6. Click Next on the Introduction Page

  7. Choose Windows as the Run As Account Type and PDWWatcher as the Display Name. Click Next.

    Click Next on Introduction page

  8. Provide the domain credentials to access the ODBC connections on the local SCOM Monitoring Server, click Next. This may be an admin account in your environment or you may create a special account with the correct access rights in order to monitor the ODBC folder. Please see the PDW MP documentation for more details.

    Type in account credentials

  9. Click Create on the Distribution Security Screen.

  10. Back at the Add a Run As Account, choose PDWWatcher and Click OK.

  11. Back at the Add Run as Accounts, PDWWatcher should be chosen and click Create …

  12. Click Close.

  13. In the SCOM Administration Pane, Click Profiles and then double-click Microsoft SQL Server PDW Watcher Account.

  14. Click Next on the Introduction Page of the Run As Profile wizard.

  15. Click Next on the General properties page of the Run As Profile wizard.

  16. Click Add on the Add Run As Accounts Page of the Run AS Account wizard.

    Add Run As Accounts page

  17. Choose PDWWatcher as the Run As Account and click OK.

  18. Ensure that PDWWatcher is chosen as the Run As Account and click Save.

  19. Click Close on the Completion screen of the wizard.

Step 3: Create Run-As Profile in System Center for PDW Monitor

The PDW Monitor account will be used to log into SQL Server PDW Appliances and retrieve the health information that is shown in SCOM.

  1. In the Operations Console, Click on Administration.

  2. Right Click on Accounts under Run As Configuration and choose Create Run As Account…

    Create Run As Account

  3. On the Introduction page, click Next.

  4. Choose Basic Authentication in the Run As Account Type and Type PDWMonitor in the Display name section. Click Next.

  5. Type in an account with the appropriate rights in the Account Name section and provide its password twice:

    Type in account credentials

  6. Click Create.

  7. Click Close.

  8. In the SCOM Administration pane, click Profiles and then double-click Microsoft SQL Server PDW Action Account.

    SCOM Administration Pane

  9. Click Next on the Introduction page of the Run As Profile wizard.

  10. Click Next on the General properties page of the Run As Profile wizard.

  11. Click Add on the Add Run As Accounts page of the Run AS Account wizard.

    Add Run As Accounts Page

  12. Choose PDWMonitor as the Run As Account and click OK.

    Choose PDW Monitor

  13. Ensure that PDW Monitor is chosen as the Run As Account and click Save.

    Choose PDW Monitor as Run As Account

  14. Click Close on the Completion screen of the Wizard.

Next Step

Now that you have configured the Management Packs, you are ready to start monitoring the appliance. For more information, see Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System).

See Also


HDInsight Management Tasks Using SCOM (Analytics Platform System)

[image]

HDInsight Management Tasks Using SCOM (Analytics Platform System)
See Also Send Feedback

The System Center Operation Manager Management Pack for HDInsight allows you to manage alerts, view the HDInsight topology, monitor the cluster condition, and view performance counters.

Management Pack Menu Option

Once imported, the management pack creates an additional menu in the Monitoring pane of Operations Console.

APS HDI SCOM Monitoring Menu

Here is a short description of available options:

Active Alerts

Provides information about monitor alerts that have been raised by all monitored HDInsight clusters.

Cluster Services Performance

A view with performance indicators relevant for the entire cluster. For this management pack HDSF indicators are available.

Cluster Summary

Dashboard page showing the most important health and performance indicators for all monitored clusters. This is the default page where administrators should start their every-day activities.

Clusters diagram

Provides a topology view of all HDInsight clusters that are configured for monitoring.

Host Components Performance

A view with performance indicators for each cluster node. This is the detailed level of information that a user can get about HDInsight performance.

HDFS

Grouping folder for HDFS service details.

HDFS Summary

Dashboard widgets displaying HDFS health and performance indicators.

NameNode Summary

Dashboard widgets displaying NameNode health and performance indicators.

MapReduce

Grouping folder for MapReduce service details.

JobTracker Summary

Dashboard widgets displaying JobTracker health and performance indicators.

MapReduce Summary

Dashboard widgets displaying MapReduce health and performance indicators.

Yarn

Grouping folder for Yarn service details.

ResourceManager Summary

Dashboard widgets displaying Yarn health and performance indicators.

Yarn Summary

Dashboard widgets displaying Resource Manager health and performance indicators.

Manage Active Alerts

An administrator can click Active Alerts and see critical event for all monitored HDInsight clusters in the data center or Windows Azure.

APS HDI SCOM Alerts

For selected alerts, an administrator can see details including which cluster was affected and the recommended actions in that situation.

APS HDI SCOM Alert Properties

Display Cluster Topology

The clusters diagram view shows all the monitored clusters and offers two perspectives: Service and Host. An administrator can learn how many nodes each cluster has, where the HDInsight components are deployed, and what their health is. The health monitors are rolled up from host component level through hosts (or services) up to the entire HDInsight cluster. The administrator can visualize where the problem is located.

APS HDI SCOM Topology

Cluster Summary Dashboard

The Cluster Summary is a view where the most important information is available for all monitored clusters. This includes:

  • Cluster overall health (based on health roll-up rules)

  • List of services with their health

  • List of host components (per service)

  • Live vs. Dead nodes widget

  • Total space utilization widget

APS HDI SCOM Dashboard

The management pack offers other dashboards, specific to HDInsight services, and those dashboards contain the the most relevant widgets.

Tracking Performance Counters

Performance counters are available at the cluster level (for front-end services) and at the level of each cluster node. To analyze the first set of counters, use the Cluster Services Performance, while for later ones click the Host Components Performance from the main menu. Performance counters are consumed in usual way by selecting counters and adjusting the time range on the graph.

APS HDI SCOM Tracking Performance Counters

See Also

Concepts

Import the SCOM Management Pack for HDInsight (Analytics Platform System)


Monitor Appliance Health State (Analytics Platform System)

[image]

Monitor Appliance Health State (Analytics Platform System)
See Also Send Feedback

This topic explains how to monitor the state of a SQL Server PDW appliance by using the Admin Console, or by directly querying the SQL Server PDW Dynamic Management Views.

To Monitor the Appliance State

A system administrator can use the Admin Console or the SQL Server PDW Dynamic Management Views (DMVs) to retrieve the full hierarchy of nodes, components, and software. The following diagram gives a high level understanding of the components that SQL Server PDW monitors.

Monitor Component Status By Using the Admin Console

To retrieve component status by using the Admin Console:

  1. Click on the Appliance State tab.

  2. On the Appliance State page, click on a specific node to view the node details.

    PDW Admin Console State

Monitor Component Status By Using System Views

To retrieve component status by using system views, use sys.dm_pdw_component_health_status (SQL Server PDW). For example, the following query retrieves the status for all components.

Copy image Copy Code
SELECT
   s.[pdw_node_id],
   n.[name] as [node_name],
   n.[address] ,
   g.[group_id] ,
   g.[group_name] ,
   c.[component_id] ,
   c.[component_name] ,
   s.[component_instance_id] ,
   p.[property_name] ,
   s.[property_value] ,
   s.[update_time]
FROM [sys].[dm_pdw_component_health_status] AS s
JOIN sys.dm_pdw_nodes AS n
   ON s.[pdw_node_id] = n.[pdw_node_id]
JOIN [sys].[pdw_health_components] AS c
   ON s.[component_id] = c.[component_id]
JOIN [sys].[pdw_health_component_groups] AS g
   ON c.[group_id] = g.[group_id]
JOIN [sys].[pdw_health_component_properties] AS p
   ON s.[property_id] = p.[property_id] AND s.[component_id] = p.[component_id]
WHERE p.property_name = 'Status'
ORDER BY
   s.[pdw_node_id],
   g.[group_name] ,
   s.[component_instance_id] ,
   c.[component_name] ,
   p.[property_name];

Possible values returned for the Status property are:

  • Ok

  • NonCritical

  • Critical

  • Unknown

  • Unsupported

  • Unreachable

  • Unrecoverable

To see all the properties for all components, remove the WHERE p.property_name = 'Status' clause.

The [update_time] column shows the last time the component was polled by the SQL Server PDW health agents.

Caution note Caution

Be sure to investigate the issue when a component has not been polled for 5 minutes or longer; there could be an alert that indicates an issue with the software heartbeats.

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Appliance Monitoring (Analytics Platform System)


Track Appliance Alerts (Analytics Platform System)

[image]

Track Appliance Alerts (Analytics Platform System)
See Also Send Feedback

This topic explains how to use the Admin Console and system views to track alerts in a SQL Server PDW appliance.

To Track Appliance Alerts

SQL Server PDW creates alerts for hardware and software issues that need attention. Each alert contains a title and a description of the issue.

SQL Server PDW logs alerts in the sys.dm_pdw_component_health_alerts (SQL Server PDW) DMV. The system retains a limit of 10,000 alerts and deletes the oldest alert first when the limit is exceeded.

View Alerts by Using the Admin Console

There is an Alerts tab for the PDW region, the HDI region, and for the fabric region of the appliance. After failover occurs, the failover event is included in the number of alerts on the page. There is a page for the PDW region, the HDI region, and for the fabric region of the appliance. Each Health page has a tab. To learn more about an alert, click the Health page, the Alerts tab, and then click an alert.

PDW Admin Console Alerts

On the Alerts page:

  • To view the alert history, click on the Review Alert History link.

  • To view the alert component and its current property values, click on the alert row.

  • To view details about the node that raised the alert, click on the node name.

View Alerts by Using the System Views

To view alerts by using system views, query sys.dm_pdw_component_health_active_alerts (SQL Server PDW). This DMV shows alerts that have not been corrected. For help with triaging alerts and errors, use the sys.dm_pdw_errors (SQL Server PDW) DMV.

The following example is a common query for viewing the current alerts.

Copy image Copy Code
SELECT
    aa.[pdw_node_id],
    n.[name] AS [node_name],
    g.[group_name] ,
    c.[component_name] ,
    aa.[component_instance_id] ,
    a.[alert_name] ,
    a.[state] ,
    a.[severity] ,
    aa.[current_value] ,
    aa.[previous_value] ,
    aa.[create_time] ,
    a.[description]
FROM [sys].[dm_pdw_component_health_active_alerts] AS aa
    INNER JOIN sys.dm_pdw_nodes AS n
        ON aa.[pdw_node_id] = n.[pdw_node_id]
    INNER JOIN [sys].[pdw_health_components] AS c
        ON aa.[component_id] = c.[component_id]
    INNER JOIN [sys].[pdw_health_component_groups] AS g
        ON c.[group_id] = g.[group_id]
    INNER JOIN [sys].[pdw_health_alerts] AS a
        ON aa.[alert_id] = a.[alert_id] and aa.[component_id] = c.[component_id]
ORDER BY
    a.alert_id ,
    aa.[pdw_node_id];

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Appliance Monitoring (Analytics Platform System)


View Capacity Utilization (Analytics Platform System)

[image]

View Capacity Utilization (Analytics Platform System)
See Also Send Feedback

This topic explains how to view capacity utilization in the SQL Server PDW appliance.

To View Capacity Utilization By Using Admin Console

To view the space used, open the Admin Console, and click the Storage tab. There is a Storage tab for both the PDW region and the HDI region.

PDW Admin Console Storage

To View Capacity Utilization By Using Queries

To understand if a node is running low on space, the SQL Server PDW health monitoring system already monitors the free space for all volumes within each node.

If the free space within a volume drops below 30%, SQL Server PDW generates a warning alert in sys.dm_pdw_component_health_active_alerts (SQL Server PDW). The alert remains until free space is made available.

If the free space within a volume drops under 10%, SQL Server PDW generates a critical alert. This is considered critical because queries could fail if they cause the database to expand.

To retrieve volume usage, see the following example.

Copy image Copy Code
SELECT
    space.[pdw_node_id] ,
    space.[node_name] ,
    MAX(space.[volume_name]) AS 'volume_name' ,
    MAX(space.[volume_size_mb]) AS 'volume_size_mb' ,
    MAX(space.[free_space_mb]) AS 'free_space_mb' ,
   (MAX(space.[volume_size_mb]) - MAX(space.[free_space_mb])) AS 'space_utilized'
FROM (
    SELECT
        s.[pdw_node_id],
        n.[name] AS [node_name],
        (CASE WHEN p.property_name = 'volume_name'
            THEN s.[property_value] ELSE NULL END) AS 'volume_name' ,
        (CASE WHEN p.property_name = 'volume_size'
            THEN (CAST(ISNULL(s.[property_value], '0') AS BIGINT)/1024/1024)
            ELSE 0 END) AS 'volume_size_mb' ,
        (CASE WHEN p.property_name = 'volume_free_space'
            THEN (CAST(ISNULL(s.[property_value], '0') AS BIGINT)/1024/1024)
            ELSE 0 END) AS 'free_space_mb' , s.[component_instance_id]
    FROM [sys].[dm_pdw_component_health_status] AS s
        JOIN sys.dm_pdw_nodes AS n
            ON s.[pdw_node_id] = n.[pdw_node_id]
        JOIN [sys].[pdw_health_components] AS c
            ON s.[component_id] = c.[component_id]
        JOIN [sys].[pdw_health_component_properties] AS p
            ON s.[property_id] = p.[property_id] AND s.[component_id] = p.[component_id]
    WHERE
        c.[Component_name] = 'Volume'
        AND p.[property_name] IN ('volume_name', 'volume_free_space', 'volume_size')
    ) AS space
GROUP BY space.[pdw_node_id] , space.[node_name] , space.[component_instance_id]
ORDER BY space.[pdw_node_id], MAX(space.[volume_name]);

To retrieve the space used by databases across the appliance nodes, see the following example.

Copy image Copy Code
SELECT
    [pdw_node_id],
    [db_name],
    SUM(CASE WHEN [file_type] = 'DATA' THEN [value_MB] ELSE 0 END) AS [DataSizeMB],
    SUM(CASE WHEN [file_type] = 'LOG' THEN [value_MB] ELSE 0 END) AS [LogSizeMB],
    SUM([value_MB]) AS [TotalMB]
FROM (
    SELECT
        pc.[pdw_node_id],
        RTRIM(pc.[counter_name]) AS [counter_name],
        ISNULL(d.[name], pc.[instance_name]) AS [db_name],
        pc.[cntr_value]/1024 AS [value_MB],
        CASE
            WHEN [counter_name] LIKE 'Data File(s) Size%'
            THEN 'DATA'
            ELSE 'LOG'
        END AS [file_type]
    FROM sys.dm_pdw_nodes_os_performance_counters AS pc
        LEFT JOIN sys.pdw_database_mappings AS dm
            ON pc.instance_name = dm.physical_name
        INNER JOIN sys.databases AS d
            ON d.database_id = dm.database_id
    WHERE
        ([counter_name] LIKE 'Log File(s) Size%'
             OR [counter_name] LIKE 'Data File(s) Size%')
        AND (d.[name] <> dm.[physical_name]
             OR pc.[instance_name] = 'tempdb')
) AS db
GROUP BY [pdw_node_id], [db_name]
ORDER BY [db_name], [pdw_node_id];

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Appliance Monitoring (Analytics Platform System)


Determine Polling Frequency (Analytics Platform System)

[image]

Determine Polling Frequency (Analytics Platform System)
See Also Send Feedback

This topic explains how to determine the polling frequency for SQL Server PDW appliance alerts.

To Determine the Polling Frequency

Since PDW does not currently support proactive notifications when alerts occur, the monitoring solution needs to continuously poll the appliance DLLs. Internally, PDW polls the components at different intervals:

  • Cluster – 60 seconds

  • Heartbeat – 60 seconds

  • All other components – 5 minutes

  • Performance counters – 3 seconds

A common interval to poll for alerts, which is also used by System Center, is every 15 minutes. Obviously, you could query more or less frequently, but it is not recommended to poll less than every 6 hours.

Polling more frequently is acceptable, but polling too frequently can clutter the sys.dm_pdw_nodes_exec_requests DMV. This can make it difficult for users to diagnose query performance issues if there query quickly rolls out of view.

See Also

Concepts

Common Metadata Query Examples (SQL Server PDW)
Appliance Monitoring (Analytics Platform System)


Determine Which Cluster Node Failed (Analytics Platform System)

[image]

Determine Which Cluster Node Failed (Analytics Platform System)
Send Feedback

This topic describes how to determine the name of the SQL Server PDW node that failed after a cluster failover has occurred and a cluster failover alert has been raised. As part of troubleshooting a cluster failover, you must determine the name of the node that failed before contacting Microsoft to help resolve the problem.

Contents

  • Background

  • Admin Console Solution

  • System View Solution

Background

For high availability in SQL Server PDW, the Control node and the Compute nodes are configured as active or passive components of Windows failover clusters. When an active server fails to respond to critical system requests, the passive server fails over and performs the functions of the server that failed.

After a cluster failover, when SQL Server PDW reports on node status, the passive server has a failed over status. However, it is not obvious which server or node failed, especially if the server that failed is still online. To troubleshoot the cluster failure, you must determine the name of the node that failed over.

[TOP]

Admin Console Solution

To find the name of the node that failed

  • Open the Admin Console. For more information about the Admin Console, see Monitor the Appliance by Using the Admin Console (Analytics Platform System). After failover occurs, the failover event is included in the number of alerts on the HEALTH page. There is an HEALTH page for the PDW region, the HDI region, and for the fabric region of the appliance. Each Health page has an ALERTS tab. To learn more about an alert, click the Health page, the Alerts tab, and then click an alert.

[TOP]

System View Solution

The following SQL statement shows how to use the sys.dm_pdw_component_health_active_alerts (SQL Server PDW) system view to find the name of the server that failed.

Copy image Copy Code
SELECT
SUBSTRING( component_instance_id, 2, charindex(' ', component_instance_id, 1)-2) AS failed_node_name,
create_time AS failover_time
FROM sys.dm_pdw_component_health_active_alerts
WHERE alert_id = 500139
ORDER BY failed_node_name;

[TOP]


Understanding Admin Console Alerts (Analytics Platform System)

[image]

Understanding Admin Console Alerts (Analytics Platform System)
See Also Send Feedback

Alerts appear in the appliance Admin Console and in System Center Operations Manager (SCOM). Use this list of alerts to help identify which alerts require additional investigation.

For information about connecting to the Admin Console by using Internet Explorer, see Monitor the Appliance by Using the Admin Console (Analytics Platform System). For information about SCOM, see Monitor the Appliance by Using System Center Operations Manager (Analytics Platform System)

For information about obtaining alert information by using Transact-SQL, see Monitor the Appliance by Using System Views (Analytics Platform System).

Types of Alerts

Alert names that indicate a NORMAL status do not usually require investigation. Alert names that contain the words NON_CRITICAL sometimes require action. Investigation is required for all other types of alerts.

Alert List

Alerts are listed alphabetically by the Alert Name. All possible alerts are not in the list. The wording of some alerts varies slightly for different vendors.

Alert Name | Action Required? | State | Severity | Description | More Information
Ambari Agent has CRITICAL status. | Yes | Failed | Error | This Ambari Agent resource has failed (status: 4) or is offline (status: 3). Other offline states included are when an offline is pending (status: 130). Status is reported in the component's "hadoop_service_status" property. | Review the Cluster Resource on the Head and Data nodes.
Ambari Agent has NON-CRITICAL status. | Yes | Degraded | Warning | This Ambari Agent resource is in an non-critical state due to one of the following reasons: - resource is in inherited state (status: 0) - resource is in pending state (status: 128) - resource is in online pending state (status: 129) - resource is performing initialization (status: 1) Status is reported in the component's "hadoop_service_status" property. | Review the Cluster Resource on the Head and Data nodes.
Ambari Agent has NORMAL status. | No | Operational | Informational | The Ambari Agent is running normally (status: Running). Status is reported in the component's "hadoop_service_status" property. |
Ambari Agent has UNKNOWN status. | Yes | Degraded | Warning | Status of this Ambari Agent resource could not be determined (status: -1). Status is reported in the component's "hadoop_service_status" property. | Review the Cluster Resource on the Head and Data nodes.
Application Heartbeat has NORMAL status. | No | Operational | Informational | Successfully established communication with the application. | Indicates that the component previously reported a different status, but has since returned to normal.
Application Heartbeat is throwing CRITICAL alert. | Yes | Non-operational | Error | Could not communicate with the application. Application might be in the process of restarting. | The application heartbeat is in an unexpected state. Troubleshooting required. Review the node's Windows event log for details.
Cluster Failover event has occurred. | Yes | Operational | Error | The primary clustered node is no longer active so the passive has taken over as the primary node. Review the failed node's Windows event log for details and review the Failover Cluster Manager on the HST01 VM. | Failover has occurred. Troubleshooting required. Review the Failover Cluster Manager on the HST01 VM and the node's system event log.
Cluster resource group has CRITICAL status. | Yes | Failed | Error | This cluster resource group has failed and may be in the process of attempting a restart, or is being offline HST01 VM. | The resource group status has failed and requires troubleshooting. Review the Failover Cluster Manager on the HST01 VM.
Cluster resource group has NON-CRITICAL status. | Yes | Degraded | Warning | This cluster resource group is online but in an non-critical state due to one of the following reasons: resource group is partially online or the resource group is in pending state. | The resource group is not completely in the expected state. Troubleshooting required. Review the Failover Cluster Manager on the HST01 VM.
Cluster resource group has NORMAL status. | No | Operational | Informational | This cluster resource group is online | Indicates that the component previously reported a different status, but has since returned to normal.
Cluster resource group has UNKNOWN status. | Yes | Degraded | Warning | This cluster resource group is in an unknown state. | The system was not able to retrieve the health status of the cluster resource group. Troubleshooting required. Review the Failover Cluster Manager on the HST01 VM.
Cluster resource has CRITICAL status. | Yes | Failed | Error | This clustered resource has failed and may be attempting a restart, or is in offline state. | Cluster resource is not in the expected state. Troubleshooting required. Review the Failover Cluster Manager on the HST01 VM.
Cluster resource has NON-CRITICAL status. | Yes | Degraded | Warning | This clustered resource is in a non-critical state due to one of the following reasons: resource is in inherited state, resource is in pending state, resource is in online pending state, resource is in offline pending state, or resource is performing initialization. | Cluster resource is not in the expected state. Troubleshooting required. Review the Failover Cluster Manager on the HST01 VM.
Cluster resource has NORMAL status. | No | Operational | Informational | This clustered resource is online. | Indicates that the component previously reported a different status, but has since returned to normal.
Cluster resource has UNKNOWN status. | Yes | Degraded | Warning | Status of this clustered resource could not be determined. | The system was not able to retrieve the health state of the cluster resource. Troubleshooting required. Review the Failover Cluster Manager on the HST01 VM.
Cluster Shared Volume has CRITICAL status. | Yes | Failed | Error | This clustered shared volume resource has failed (status: 4) or is offline (status: 3). Other offline states included are when an offline is pending (status: 130). Status is reported in the component's "csv_state" property. | Review the Failover Cluster Manager on the HST01 VM.
Cluster Shared Volume has NON-CRITICAL status. | Yes | Degraded | Warning | This clustered shared volume resource is in an non-critical state due to one of the following reasons: - resource is in inherited state (status: 0) - resource is in pending state (status: 128) - resource is in online pending state (status: 129) - resource is performing initialization (status: 1) Status is reported in the component's "csv_state" property. | Review the Failover Cluster Manager on the HST01 VM.
Cluster Shared Volume has NORMAL status. | No | Operational | Informational | This clustered shared volume resource is online (status: 2). Status is reported in the component's "csv_state" property. |
Cluster Shared Volume has UNKNOWN status. | Yes | Degraded | Warning | Status of this clustered shared volume resource could not be determined (status: -1). Status is reported in the component's "csv_state" property. | Review the Failover Cluster Manager on the HST01 VM.
Cluster Status Normal | No | Operational | Informational | Cluster has NORMAL status. | Indicates that the component previously reported a different status, but has since returned to normal.
Controller has CRITICAL status. | Yes | Failed | Error | The PERC disk is indicating there is a critical error, or the controller has been powered off. | The local RAID controller has a critical error and may need to be replaced. Troubleshooting required. Review the node's Windows event log for details.
Controller has NON-CRITICAL status. | Yes If problem persists more than 7 hours or reoccurs multiple times on the same node not tied to expected reboots | Degraded | Warning | The PERC disk reported non-critical problem, probably related to cable malfunction. | This most commonly indicates a battery recharging cycle on the PowerEdge RAID Controller's battery-backed cache module. This could be the scheduled test cycle (duration up to ~7 hours)or it also could be reported after reboots or power cycles when the battery must recharge. Important: This also usually indicates that the controller's policy temporarily has changed from write-through to write-back until the charging is complete which will have performance implications on the local storage (tempdb). Review the node's Windows event log for details.
Controller has NON-RECOVERABLE status. | Yes | Failed | Error | The PERC disk status is non-recoverable. | The local RAID controller is not functional and has entered a non-recoverable state and may need to be replaced. Troubleshooting required. Review the node's Windows event log for details.
Controller has NORMAL status. | No | Operational | Informational | The PERC disk is running normally | Indicates that the component previously reported a different status, but has since returned to normal.
Controller has UNKNOWN status. | Yes | Degraded | Warning | The status of the PERC disk could not be determined. | The system could not retrieve the health state of the local RAID controller. Troubleshooting required. Review the node's Windows event log for details.
Cooling device has CRITICAL status. | Yes | Failed | Warning | The cooling device has reached critical upper or lower threshold | The cooling device may require replacement. Troubleshooting required. Review the node's Windows event log for details.
Cooling device has NON-CRITICAL status. | Yes | Degraded | Warning | The cooling device has reached non critical upper or lower threshold. | The cooling device hasn't reached critical levels, but is outside the expected upper or lower range. Review the node's Windows event log for details.
Cooling device has NON-RECOVERABLE status. | Yes | Failed | Warning | The cooling device has reached non-recoverable upper or lower threshold. | The cooling device may require replacement. Troubleshooting required. Review the node's Windows event log for details.
Cooling device has NORMAL status. | No | Operational | Informational | The cooling device is running normally. | Indicates that the component previously reported a different status, but has since returned to normal.
Cooling device has UNKNOWN status. | Yes | Degraded | Warning | The status of the cooling device could not be determined | The system could not retrieve the status of the cooling device. Troubleshooting required. Review the node's Windows event log for details.
Disk array has CRITICAL overall status. | Yes | Failed | Error | The disk array overall status is critical. | Could indicate that the disk array is no longer active due to failed drives or a similar problem. Troubleshooting required. Review the node's Windows event log for details.
Disk array has NON-CRITICAL overall status. | Yes | Degraded | Warning | The disk array overall status is indicating there is a non-critical warning but system is still operational. | The disk array is still functional, but this could indicate a disk failure or a similar problem. Troubleshooting required. Review the node's Windows event log for details.
Disk array has NON-RECOVERABLE overall status. | Yes | Failed | Error | The disk array overall status is non-recoverable. | The disk array is no longer functional. Troubleshooting required. Review the node's Windows event log for details.
Disk array has NORMAL overall status. | No | Operational | Informational | The disk array overall status is normal. | Indicates that the component previously reported a different status, but has since returned to normal.
Disk array has UNKNOWN overall status. | Yes | Degraded | Warning | Overall status of the disk array could not be determined. | The system cannot retrieve the health state of the local disk array. Troubleshooting required. Review the node's Windows event log for details.
External Storage Array has CRITICAL status. | Yes | Failed | Error | The external storage array is indicating there is a failure (Vendor OperationalStatus: 6, 16)! Vendor status is reported in the component's "storage_global_status" property. Values: 6-Error, 16-Supporting Entity Error. | Review the node's Windows event log for details or contact device manufacturer.
External Storage Array has NON-CRITICAL status. | Yes | Degraded | Warning | The external storage array reported non-critical warning (Vendor OperationalStatus: 3,4,5,11,14,15,17). Vendor status is reported in the component's "storage_global_status" property. Values: 3-Degraded, 4-Stressed, 5-Predictive Failure, 11-In Service, 14-Aborted, 15-Dormant, 17-Completed Operation. | Review the node's Windows event log for details or contact device manufacturer.
External Storage Array has NON-RECOVERABLE status. | Yes | Failed | Error | The external storage array is indicating that the storage array is down and non-recoverable (Vendor OperationalStatus: 7). Vendor status is reported in the component's "storage_global_status" property. | Review the node's Windows event log for details or contact device manufacturer.
External Storage Array has NORMAL status. | No | Operational | Informational | The external storage array is working normally (vendor status: ok). Vendor status is reported in the component's "storage_global_status" property. |
External Storage Array has UNKNOWN status. | Yes | Degraded | Warning | The status of the external storage array could not be determined based on the vendor status (Vendor OperationalStatus: 0,1,18). Vendor status is reported in the component's "storage_global_status" property. Values: 0-Unkown, 1-Other, 18-Power Mode. | Review the node's Windows event log for details or contact device manufacturer.
External Storage Array has UNREACHABLE status. | Yes | Failed | Error | The external storage array is indicating that the storage array is unreachable (Vendor OperationalStatus: 8,9,10,12,13). Vendor status is reported in the component's "storage_global_status" property. Values: 8-Starting, 9-Stopping, 10-Stopped, 12-No Contact, 13-Lost Communication. | Review the node's Windows event log for details or contact device manufacturer.
External Storage has CRITICAL status. | Yes | Failed | Error | The external storage is indicating there is a failure. | Troubleshooting required. Review Windows event log and the storage device's event log for details.
External Storage has DEGRADED status. | Yes | Degraded | Warning | The storage system is degraded. You need to check the temperature status or power supply status of this storage system. Additionally, if the side panel for the storage system is removed, the air flow changes could result in improper cooling of the drives and affect the temperature status. Vendor status is reported in the component's "storage_global_status" property. | Review the node's Windows event log for details or contact device manufacturer.
External Storage has NON-CRITICAL status. | Yes if problem persists for more than 7 hours or reoccurs frequently on the same device more than every 90 days | Degraded | Warning | The external storage reported non-critical warning. | This event typically indicates one of two issues: Disk failures/transition events or battery recharging cycles on the raid controller's battery-backed cache module. The charging cycles are usually scheduled every 90 days and can take up to 7 hours. Important: During this time it is likely that the controller's write-cache policy has temporarily changed from write-through to write-back which can affect performance. Review the Windows event log and the storage device's event log for details.
External Storage has NORMAL status. | No | Operational | Informational | The external storage is working normally. | Indicates that the component previously reported a different status, but has since returned to normal.
External Storage has UNKNOWN status. | Yes | Degraded | Warning | The status of the external storage could not be determined. | The system was not able to retrieve the health state of the server's external storage. Troubleshooting required. Review the node's Windows event log and the storage device's event log for details.
Fan device has CRITICAL status. | Yes | Failed | Warning | The fan device has reached critical upper or lower threshold (vendor status: CriticalUpper or CriticalLower). Vendor status is reported in the component's "device_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Fan device has NON-CRITICAL status. | Yes | Degraded | Warning | The fan device has reached non-critical upper or lower threshold (vendor status: nonCriticalUpper or nonCriticalLower). Vendor status is reported in the component's "device_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Fan device has NON-RECOVERABLE status. | Yes | Failed | Warning | The fan device has reached non-recoverable upper or lower threshold (vendor status: failed, nonRecoverableUpper or nonRecoverableLower). Vendor status is reported in the component's "device_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Fan device has NORMAL status. | No | Operational | Informational | The fan device is running normally (vendor status: ok). Vendor status is reported in the component's "device_status" property. |
Fan device has UNKNOWN status. | Yes | Degraded | Warning | The status of the fan device could not be determined (vendor status: other or unknown). Vendor status is reported in the component's "device_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Fibre Channel host controller has CRITICAL status. | Yes | Failed | Warning | The Fibre Channel host controller component detects one of the following conditions: - host controller has failed and should be replaced (vendor status: failed) - host controller has been shutdown (vendor status: shutdown) - the fibre channel connection is failed (vendor status: loopFailed) Vendor status is reported in the component's "FC_device_rollup_status" property. | Review the node's Windows event log for details or contact device manufacturer. User Action: If the controller status is failed, replace the controller.
Fibre Channel host controller has NON-CRITICAL status. | Yes | Degraded | Warning | The Fibre Channel host controller is reporting one of the following conditions: - fibre channel connection is degraded (vendor status: loopDegraded) - fibre channel port is not connected or the device to which it is connected is powered down (vendor status: notConnected) Vendor status is reported in the component's "FC_device_rollup_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Fibre Channel host controller has NORMAL status. | No | Operational | Informational | The Fibre Channel host controller is operating normally (vendor status: ok). Vendor status is reported in the component's "FC_device_rollup_status" property. |
Fibre Channel host controller has UNKNOWN status. | Yes | Degraded | Warning | Fibre Channel host controller status could not be determined or controller is not present (vendor status: other). Vendor status is reported in the component's "FC_device_rollup_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Hadoop Service has CRITICAL status. | Yes | Non Operational | Error | This service is in a critical state and has stopped working (status: Installed or Stopped) or is in transitional state to be stopped (status: Stopping). Status is reported in the component's "hadoop_service_status" property. | Review the node's Windows and PDW Component event logs for details.
Hadoop Service has NON-CRITICAL status. | Yes | Degraded | Warning | This service is in a non critical state due to one of the following reasons: - service is starting (status: Starting) - service is upgrading (status: Upgrading) Status is reported in the component's "hadoop_service_status" property. | Review the node's Windows and PDW Component event log for details.
Hadoop Service has UNKNOWN status. | Yes | Degraded | Warning | This service is reporting that it is in a unknown state. Status is reported in the component's "hadoop_service_status" property. | Review the node's Hadoop logs, plus the Windows and PDW Component event log for details.
Memory device has CRITICAL status. | Yes | Failed | Warning | The memory is reporting critical problem. | A DIMM may need to be replaced. Troubleshooting required. A server may still be active with some failed RAM but performance may be affected. Review the node's Windows event log for details.
Memory device has NON-CRITICAL status. | Yes | Degraded | Warning | The memory is reporting non-critical situation. | Could point to imminent DIMM failure. Generally this means the DIMM has seen errors, but it is not yet past the threshold to make it a critical/failed status. A server may still be active with some failed ram, but performance may be affected. Hardware log must be cleared to clear the error. Review the node's Windows event log for details.
Memory device has NON-RECOVERABLE status. | Yes | Failed | Warning | The memory reported non recoverable problem. | A DIMM may need to be replaced. Troubleshooting required. A server may still be active with some failed ram, but performance may be affected. Review the node's Windows event log for details
Memory device has NORMAL status. | No | Operational | Informational | The memory is working normally | Indicates that the component previously reported a different status, but has since returned to normal.
Memory device has UNKNOWN status. | Yes | Degraded | Warning | Status of the memory could not be determined. | The system cannot retrieve the health state of the system memory. A DIMM may need to be replaced. Troubleshooting required. A server may still be active with some failed RAM but performance may be affected. Review the node's Windows event log for details
Network adapter has CRITICAL status. | Yes | Degraded | Warning | The network adapter is raising critical alert due to one of the following reasons: adapter is offline, adapter has been powered off, or adapter is in off duty status | The network adapter is in a failed state and may need replacement (which could mean motherboard replacement). Troubleshooting required. Review the node's Windows event log for details.
Network adapter has NON-CRITICAL status. | Yes | Degraded | Warning | The network adapter is indicating there is a non-critical warning but is still operational, potentially degrading performance though. | The network adapter has some errors, but is not in a critical state. Since this could affect performance troubleshooting is required. Review the node's Windows event log for details.
Network adapter has NON-RECOVERABLE status. | Yes | Failed | Warning | The network adapter is in non-recoverable status due to potentially being installed in error. | The network adapter is in a failed state and may need replacement (which could mean motherboard replacement). Troubleshooting required. Review the node's Windows event log for details.
Network adapter has NORMAL status. | No | Operational | Informational | The network adapter is online and running normally. | Indicates that the component previously reported a different status, but has since returned to normal.
Network adapter has UNKNOWN status. | Yes | Degraded | Warning | The status of the network adapter could not be determined. This status could be caused due to one of the following reasons: - network adapter is in Power Save mode: standby, low power, warning, unknown, or power cycle, network adapter has not been installed, network adapter device reported unknown status, network adapter might be in testing state. | The system was not able to retrieve the health state of the network adapter. Troubleshooting required. Review the node's Windows event log for details.
Network connection has CRITICAL status. | Yes | Degraded | Warning | The network connectivity is raising critical alert due to one of the following reasons: network is disconnected, hardware is not present, hardware has been disabled, media is disconnected,- authentication has failed, invalid address was used, credential is required but not supplied | The network adapter is in a critical state. Review the node's Windows event log for details.
Network connection has NON-CRITICAL status. | Yes | Degraded | Warning | The network is reporting a non-critical state. This status could be due to one of the following reasons: network is in connecting state, network is disconnecting state, network authentication is in process. | The network adapter is in an unexpected state. If this problem persists or happens multiple times then troubleshooting is required. Review the node's Windows event log for details.
Network connection has NORMAL status. | No | Operational | Informational | The network is connected and working correctly. | Indicates that the component previously reported a different status, but has since returned to normal.
Network connection profile is on a expected profile. | No | Operational | Informational | The network is connected and working as an expected profile. The profile is reported in the component's "profile_category" property. Domain profile is 2 and Private profile is 1. | Review the node's events in log 'Application and service logs\Microsoft\Windows\StorageSpaces-Driver\Operational' for further details. The health of the mirror could be impacted by the loss of a single disk so another alert might have occurred for the disk itself.
Network connection profile is showing to be on the Public profile. | Yes | Degraded | Warning | The network is reporting that it is on the Public profile. The profile is reported in the component's "profile_category" property. Public profile is reported as 0. This could cause communication issues for this node. | Review the node's Windows event log for details or contact device manufacturer.
Node in a cluster has CRITICAL status. | Yes | Failed | Error | The clustered node is down. | A server in the cluster is down. Review the Failover Cluster Manager on the HST01 VM.
Node in a cluster has NON-CRITICAL status. | Yes | Degraded | Warning | The clustered node is throwing a non-critical alert. One of the following situations might have occurred: node is in paused state or node is in the process of joining the cluster. | The node is in an unexpected state. Troubleshooting required. Review the Failover Cluster Manager on the HST01 VM.
Node in a cluster has NORMAL status. | No | Operational | Informational | The clustered node is up and running | Indicates that the component previously reported a different status, but has since returned to normal.
Node in a cluster has UNKNOWN status. | Yes | Degraded | Warning | The clustered node is in an unknown state. | The system was not able to retrieve the health state of the node. Troubleshooting required. Review the Failover Cluster Manager on the HST01 VM.
Physical Disk has CRITICAL status. | Yes | Failed | Error | The disk status is critical (vendor status: 2-Unhealthy)! The status is reported in the component's "phys_disk_status" property. The Operational Status, shown in property "phys_disk_oper_status" might provide more information about the problem. Operational Status Values: 0-The operational status of the physical disk is unknown. 2-OK 3-Degraded 4-Stressed 5-Predictive Failure 6-Error 7-Non-Recoverable Error 8-Starting 9-Stopping 10-Stopped 11-In Service 12-No Contact 13-Lost Communication 15-Dormant 18-Power Mode 0x8004-Failed Media 0x8005-Split 0x8006-Stale Metadata 0x8007-IO Error 0x8008-Corrupt Metadata. |
Physical Disk has NON-CRITICAL status. | Yes | Degraded | Warning | The disk status is indicating there is a non-critical warning but system is still operational. The status is reported in the component's "phys_disk_status" property. The Operational Status, shown in property "phys_disk_oper_status" might provide more information about the problem. Operational Status Values: 0-The operational status of the physical disk is unknown. 2-OK 3-Degraded 4-Stressed 5-Predictive Failure 6-Error 7-Non-Recoverable Error 8-Starting 9-Stopping 10-Stopped 11-In Service 12-No Contact 13-Lost Communication 15-Dormant 18-Power Mode 0x8004-Failed Media 0x8005-Split 0x8006-Stale Metadata 0x8007-IO Error 0x8008-Corrupt Metadata. | Review the node's events in log 'Application and service logs\Microsoft\Windows\StorageSpaces-Driver\Operational' for further details. The health of the mirror could be impacted by the loss of a single disk so another alert might have occurred for the disk itself.
Physical Disk has NORMAL status. | No | Operational | Informational | The disk status is normal. The status is reported in the component's "phys_disk_status" property. |
Physical Disk has UNKNOWN status. | Yes | Degraded | Warning | The disk status could not be determined (status: 5-Unknown). The status is reported in the component's "phys_disk_status" property. The Operational Status, shown in property "phys_disk_oper_status" might provide more information about the problem. Operational Status Values: 0-The operational status of the physical disk is unknown. 2-OK 3-Degraded 4-Stressed 5-Predictive Failure 6-Error 7-Non-Recoverable Error 8-Starting 9-Stopping 10-Stopped 11-In Service 12-No Contact 13-Lost Communication 15-Dormant 18-Power Mode 0x8004-Failed Media 0x8005-Split 0x8006-Stale Metadata 0x8007-IO Error 0x8008-Corrupt Metadata. | Review the node's events in log 'Application and service logs\Microsoft\Windows\StorageSpaces-Driver\Operational' for further details.
Power Supply has CRITICAL status. | Yes | Failed | Warning | The power supply is indicating there is a critical error. | The power supply may require replacement. Troubleshooting required. Power supplies are redundant so the server may still be active. Review the node's Windows event log for details.
Power Supply has NON-CRITICAL status. | Yes | Operational | Warning | The power supply reported non-critical problem. | The power supply has reported a problem, but is not in a failed state. This might indicate imminent failure. Power supplies are redundant so a failure might not create a server outage. A hardware error probably needs to be cleared to clear the admin console error. Review the node's Windows event log for details.
Power supply has NON-RECOVERABLE status. | Yes | Failed | Warning | The power supply is in non-recoverable status. | The power supply may require replacement. Troubleshooting required. Power supplies are redundant so the server may still be active. Review the node's Windows event log for details.
Power Supply has NORMAL status. | No | Operational | Informational | The power supply is running normally. | Indicates that the component previously reported a different status, but has since returned to normal.
Power supply has UNKNOWN status. | Yes | Degraded | Warning | The status of the power supply could not be determined. | The system was not able to retrieve the health state of the power supply. Power supplies are redundant so the server may still be active. Troubleshooting required. Review the node's Windows event log for details.
Processor device has CRITICAL status. | Yes | Failed | Warning | The CPU is reporting critical problem. | CPU may need to be replaced. Troubleshooting required. Review the node's Windows event log for details.
Processor device has NON-CRITICAL status. | Yes | Degraded | Warning | The CPU is reporting non critical situation. | The CPU encountered an error, but is not yet in a failed state. This may indicate an imminent failure. Review the node's Windows event log for details.
Processor device has NON-RECOVERABLE status. | Yes | Failed | Warning | The CPU reported non recoverable problem. | Similar to critical status. CPU may need to be replaced. Troubleshooting required. Review the node's Windows event log for details.
Processor device has NORMAL status. | No | Operational | Informational | The CPU is working normally. | Indicates that the component previously reported a different status, but has since returned to normal.
Processor device has UNKNOWN status. | Yes | Degraded | Warning | Status of the CPU could not be determined. | The system cannot retrieve the health state of a CPU and further investigation is required. Review the node's Windows event log for details.
SAS Host Bus Adapter has DEGRADED condition. | Yes | Degraded | Warning | The SAS Host Bus Adapter is reporting that the overall condition of the HBA and all of the physical drives controlled by it is degraded (vendor status: degraded). Vendor status is reported in the component's "hba_device_status" property. | Review the node's Windows event log for details or contact device manufacturer.
SAS Host Bus Adapter has FAILED condition. | Yes | Failed | Warning | The SAS Host Bus Adapter is reporting that the overall condition of the HBA is in a failed state, including all of the physical drives controlled by it. This will require a component to be replaced (vendor status: failed). Vendor status is reported in the component's "hba_device_rollup_status" property. | Review the node's Windows event log for details or contact device manufacturer.
SAS Host Bus Adapter has NORMAL status. | No | Operational | Informational | The SAS Host Bus Adapter is operating normally (vendor status: ok). Vendor status is reported in the component's "hba_device_rollup_status" property. |
SAS Host Bus Adapter has UNKNOWN status. | Yes | Degraded | Warning | The SAS Host Bus Adapter status could not be determined (vendor status: other). Vendor status is reported in the component's "hba_device_status" property. | Review the node's Windows event log for details or contact device manufacturer.
SQL Server has CRITICAL status. | Yes | NonOperational | Error | This service is in a critical state and has stopped working (status: Stopped) or is in transitional state to be stopped (status: StopPending). Status is reported in the component's "sql_server_service_status" property. | Review the node's Windows event log for details.
SQL Server has NORMAL status. | No | Operational | Informational | This service is running normally (status: Running). Status is reported in the component's "sql_server_service_status" property. |
Storage Enclosure Fan has a DEGRADED status. | Yes | Degraded | Warning | The Storage Enclosure Fan is reporting that it is degraded (vendor status: 10,15). Vendor status is reported in the component's "storage_fan_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Storage Enclosure Fan has a FAILED status. | Yes | Failed | Warning | The Storage Enclosure Fan is reporting that it is in a failed state. This will require a component to be replaced (vendor status: 20,25). Vendor status is reported in the component's "storage_fan_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Storage Enclosure Fan has a NON-RECOVERABLE status. | Yes | Failed | Warning | The Storage Enclosure Fan is reporting that this fan is in a non-recoverable state. This will require a component to be replaced (vendor status: 30). Vendor status is reported in the component's "storage_fan_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Storage Enclosure Fan has a UNKOWN status. | Yes | Degraded | Error | The Storage Enclosure Fan status could not be determined (vendor status: 0-unknown). Vendor status is reported in the component's "storage_fan_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Storage Enclosure Fan has a NORMAL status. | No | Operational | Informational | The Storage Enclosure Fan is operating normally (vendor status: 5). Vendor status is reported in the component's "storage_fan_status" property. |
Storage Enclosure Power Supply has a DEGRADED status. | Yes | Degraded | Warning | The Storage Enclosure Power Supply is reporting that this power supply is degraded (vendor status: 10,15). Vendor status is reported in the component's "storage_power_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Storage Enclosure Power Supply has a FAILED status. | Yes | Failed | Error | The Storage Enclosure Power Supply is reporting that this power supply is in a failed state. This will require a component to be replaced or restore power to the device (vendor status: 20,25). Vendor status is reported in the component's "storage_power_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Storage Enclosure Power Supply has a NON-RECOVERABLE status. | Yes | Failed | Error | The Storage Enclosure Power Supply is reporting that this power supply is in a non-recoverable state. This will require a component to be replaced (vendor status: 30). Vendor status is reported in the component's "storage_power_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Storage Enclosure Power Supply has a UNKNOWN status. | Yes | Degraded | Warning | The Storage Enclosure Power Supply status could not be determined (vendor status: 0). Vendor status is reported in the component's "storage_power_status" property. | Review the node's Windows event log for details or contact device manufacturer.
Storage Enclosure Power Supply has a NORMAL status. | No | Operational | Informational | The Storage Enclosure Power Supply is operating normally (vendor status: 5). Vendor status is reported in the component's "storage_power_status" property. |
Storage Pool has CRITICAL status. | Yes | Failed |  | The storage pool status is critical (vendor status: 2-Unhealthy)! The status is reported in the component's "storage_pool_status" property. The Operational Status, shown in property "storage_pool_oper_status" might provide more information about the problem. | Review the node's events in log 'Application and service logs\Microsoft\Windows\StorageSpaces-Driver\Operational' for further details. The health of the mirror could be impacted by the loss of a single disk so another alert might have occurred for the disk itself.
Storage Pool has NON-CRITICAL status. | Yes | Degraded |  | The storage pool status is indicating there is a non-critical warning but system is still operational (status: 1-Warning). The status is reported in the component's "storage_pool_status" property. The Operational Status, shown in property "storage_pool_oper_status" might provide more information about the problem. | Review the node's events in log 'Application and service logs\Microsoft\Windows\StorageSpaces-Driver\Operational' for further details.. The health of the mirror could be impacted by the loss of a single disk so another alert might have occurred for the disk itself.
Storage Pool has NORMAL status. | No | Operational |  | The storage pool status is normal (status: 0-Healthy). The status is reported in the component's "storage_pool_status" property. |
Storage Pool has UNKNOWN status. | Optional | Operational |  | The storage pool status is in an unknown state on this node (status: 5-Unknown). The status is reported in the component's "storage_pool_status" property. The Operational Status, shown in property "storage_pool_oper_status" might provide more information about the problem. This commonly happens when the node querying the storage pool state is not the owner of the storage pool. | Review the node's events in log 'Application and service logs\Microsoft\Windows\StorageSpaces-Driver\Operational' for further details.
Temperature status is CRITICAL. | Yes | Failed | Error | The temperature has reached critical upper or lower threshold. | The temperature is too high or too low. Continuing at this state could damage or drastically shorten the life of the hardware. Troubleshooting required. Review the node's Windows event log for details.
Temperature status is NON-CRITICAL. | Optional | Degraded | Warning | The temperature has reached non critical upper or lower threshold | The temperature reported by the server is at a level higher or lower than normal, but has not reached the threshold for critical status. Temperatures outside of threshold shorten hardware life. Things that could affect temperature are workload, data center temperature/airflow, cabling restricting server exhaust, etc. Review the node's Windows event log for details.
Temperature status is NON-RECOVERABLE. | Yes | Failed | Warning | The temperature is in non-recoverable status. | The temperature sensor has detected an error from which it cannot recover. This could be a problem with the temperature or with the temperature module itself. Review the node's Windows event log for details.
Temperature status is NORMAL. | No | Operational | Informational | The temperature is normal | Indicates that the component previously reported a different status, but has since returned to normal.
Temperature status is UNKNOWN. | Yes | Degraded | Warning | The status of the temperature could not be determined. | The system was unable to retrieve the server temperature. Troubleshooting required. Review the node's Windows event log for details.
Virtual Disk has CRITICAL status. | Yes | Failed | Error | The storage spaces virtual disk status is critical (vendor status: 2-Unhealthy)! The status is reported in the component's "virtual_disk_status" property. The Operational Status, shown in property "virtual_disk_oper_status" might provide more information about the problem. | Review the node's events in log 'Application and service logs\Microsoft\Windows\StorageSpaces-Driver\Operational' for further details. The health of the mirror could be impacted by the loss of a single disk so another alert might have occurred for the disk itself.
Virtual Disk has NON-CRITICAL status. | Yes | Degraded | Warning | The storage spaces virtual disk status is indicating there is a non-critical warning but system is still operational (status: 1-Warning). The status is reported in the component's "virtual_disk_status" property. The Operational Status, shown in property "virtual_disk_oper_status" might provide more information about the problem. If the Virtual Disk has moved to another node, then review the state of the cluster shared volume components and move the disks back to the expected owner, indicated by the number after the N in the name, eg. N01D01 belongs on HSA01. | Review the node's events in log 'Application and service logs\Microsoft\Windows\StorageSpaces-Driver\Operational' for further details. The health of the mirror could be impacted by the loss of a single disk so another alert might have occurred for the disk itself.
Virtual Disk has NORMAL status. | No | Operational | Informational | The storage spaces virtual disk status is normal (status: 0-Healthy). The status is reported in the component's "virtual_disk_status" property. |
Virtual Disk has UNKNOWN status. | Yes | Operational | Warning | The storage spaces virtual disk status could not be determined (status: 5-Unknown). The status is reported in the component's "virtual_disk_status" property. The Operational Status, shown in property "virtual_disk_oper_status" might provide more information about the problem. If the Virtual Disk has moved to another node, then review the state of the cluster shared volume components and move the disks back to the expected owner, indicated by the number after the N in the name, eg. N01D01 belongs on HSA01. | Review the node's events in log 'Application and service logs\Microsoft\Windows\StorageSpaces-Driver\Operational' for further details.
Volume free space status is CRITICAL. | Yes | Degraded | Error | Volume free space is critically low! The current volume used disk space is beyond 90% of total capacity. Clean up unnecessary files/data to ensure normal appliance operation. | The Admin Console reports allocated space and not necessarily used space. You can use DBCC PDW_SHOWSPACEUSED to investigate used vs. allocated space. You can also use DBCC SHRINKLOG (SQL Server PDW) or ALTER DATABASE (SQL Server PDW) to shrink databases.
Volume free space status is NON-CRITICAL. | Optional | Operational | Warning | The current volume used disk space is between 70% and 90% full. Review disk space used on this volume and clean up unnecessary files/data to ensure normal appliance operation. | The Admin Console reports allocated space and not necessarily used space. You can use DBCC PDW_SHOWSPACEUSED (SQL Server PDW) to investigate used vs. allocated space. You can also use DBCC SHRINKLOG (SQL Server PDW) or ALTER DATABASE (SQL Server PDW) to shrink databases.
Volume free space status is NORMAL. | No | Operational | Informational | There is enough free disk space on this volume. The current volume used disk space is below 70%. | Indicates that the component previously reported a different status, but has since returned to normal.

See Also

Concepts

Error Messages (SQL Server PDW)


Getting SQL Server Parallel Data Warehouse Assistance

[image]

Getting SQL Server Parallel Data Warehouse Assistance
Send Feedback

There are three principal sources of information from Microsoft about SQL Server PDW:

  • The documentation installed with SQL Server PDW.

  • The SQL Server sites on the Microsoft Developer Network (MSDN) and TechNet.

  • SQL Server PDW information on additional Microsoft Web sites.

If you do not find the information you are looking for in these sources, you can either post questions in the SQL Server community forums and newsgroups, or you can open a support case with Microsoft.

SQL Server Documentation

When you install SQL Server PDW, you can install the SQL Server PDW documentation, which covers the concepts and procedures required to use SQL Server effectively. It also includes reference material for the languages and programming interfaces for storing, retrieving, reporting, and modifying data using SQL Server PDW.

Information on MSDN and TechNet

The Microsoft Developer Network (MSDN) provides online and offline services that help developers write applications using Microsoft products and technologies. The Microsoft TechNet Program delivers comprehensive technical resources that help IT professionals efficiently evaluate, deploy, and support Microsoft solutions.

Several sites on MSDN and TechNet provide a specific audience with links to the SQL Server information most relevant to their jobs, including the information in the copies of the SQL Server Books Online on MSDN or TechNet. These sites also contain information that extends the information available in the Books Online.

The following table describes these resources and their intended audience.

Resource | Audience
SQL Server Developer Center | Developers who are building SQL Server databases, reporting systems, and business intelligence solutions, and enabling their applications and Web sites to use these systems.
Data Access and Storage Developer Center | Developers who are enabling their applications and Web sites to use data stored using products such as SQL Server and Access.
XML Developer Center | Web site creators and application developers whose sites and applications utilize XML for data storage and manipulation.
SQL Server TechNet Site | Administrators and operators who plan, deploy, administer, and support systems that use SQL Server.

Additional Online Information

Additional SQL Server information is available from these Microsoft Web sites.

Resource | Description
Microsoft SQL Server | Provides pre-sales information about evaluating SQL Server as a data storage and business intelligence tool.
Microsoft Knowledge Base | Provides a searchable repository of SQL Server articles written by Microsoft support.
Microsoft Learning | Describes Microsoft training and certifications for SQL Server.
Microsoft Press | Lists Microsoft Press books about SQL Server.

Getting Assistance From Others

If you have not found the information you are looking for in the product documentation or on the Web you can ask a question in the SQL Server community forums and newsgroups. You can also request help from product support.

Note Note

Please do not post questions in the Community Comments section at the bottom of the SQL Server Books Online topics in the MSDN Library or TechNet Library. Those sections are not monitored for questions, and you will usually not receive an answer. Please post questions in the community forums and newsgroups, which are monitored by other customers who can answer your questions.

The following table links to and describes these resources.

Resource | Description
SQL Server IT Professional Community | Has links to newsgroups and forums monitored by the SQL Server community. It also lists community information sources, such as blogs and Web sites. The SQL Server community is very helpful in answering questions, although there is no guarantee of an answer.
SQL Server Developer Center Community | This site focuses on the newsgroups, forums, and other community resources that are useful to SQL Server developers.
Microsoft Help and Support | You can use this Web site to open a support case with a Microsoft support professional.

Microsoft products have a support lifecycle that defines how long support is available for each release. Microsoft support maintains a page that defines the Microsoft support lifecycle. The page contains links to other pages that specify the availability of support for Microsoft products. For more information, see Microsoft Support Lifecycle.