Search This Blog

Showing posts with label SQL 2008. Show all posts
Showing posts with label SQL 2008. Show all posts

EXECUTE master.dbo.xp_delete_file in Sql Server 2008

This is a quick post to show some details on this undocumented stored procedure used to delete older backup files. Parameters for this stored procedure are not documented in BoL or not readily available, may be for a good reason. But I wanted to add some details to shed some light.
1
2
3
4
5
6
7
8
EXECUTE master.dbo.xp_delete_file
   0
 , N'S:\SqlMaintFolder\'
 , N'bak'
 , N'2012-10-10T01:05:34'
 , 1
 
 
Some points to note:
  1. First parameter is ‘FileTypeSelected’, it says what kind of files to delete; 1 means Report Files, 0 means Backup files.
  2. Second parameter indicates, the path where the files are located. Make sure there is a trailing ‘\’
  3. Third parameter indicates the file extension. Make sure there isno dot in the file extension  [ '.bak' ]
  4. Fourth parameter: delete all files before this date and time.
  5. Fifth parameter indicates if you want to delete files from sub-folders or not.

SQL Server 2008: Backup Compression - Part 2


Data compression is an Enterprise Edition only feature that was added in SQL Server 2008. It allows you to use either Page or Row compression on the clustered index or any non-clustered indexes on a table. Data compression lets SQL Server trade off disk space and I/O pressure for some extra CPU pressure. It is best suited to relatively large tables that that are relatively static. Small tables that are highly volatile are not usually good candidates for compression (especially page compression).

I thought I would do a simple experiment to measure the effects of page compression on a table with 2.3 million rows of data. I created four identical tables with the schema shown below. The EventLog table has a non-compressed clustered index, the EventLogCompressed table has a compressed clustered index, the EventLogNew table has a non-compressed clustered index and a non-compressed non-clustered index, and the EventLogNewCompressed table has a compressed clustered index and a compressed non-clustered index. Each table has the exact same data. I wrote four identical stored procedures that hit each of the four tables.

This stored procedure causes a clustered index scan on the tables that don’t have a non-clustered index (EventLog and EventLogCompressed), and an index seek on the tables that have the non-clustered index (EventLogNew and EventLogNewCompressed). The tables with Compressed in their name have page compression on each index in the table. For this data, page compression compressed the clustered index by about 4.5 to 1.

Page compression reduced the logical reads from 23941 to 5545 for the clustered index scan and the query cost by 66%. It also reduced the memory consumed by the clustered index scan from 186MB to 43MB. In my mind, this means that having a compressed clustered index gives you some extra protection from the effects of an expensive clustered index scan. More details about these results are shown below:


-- -- Table with 2.3 million rows
CREATE TABLE [dbo].[EventLog](
    [EventID] [bigint] IDENTITY(1,1) NOT NULL,
    [UserID] [int] NOT NULL,
    [GroupID] [int] NULL,
    [PostID] [bigint] NULL,
    [FeedID] [int] NULL,
    [EventChange] [int] NOT NULL,
    [EventObject] [int] NOT NULL,
    [EventType] [varchar](50) NOT NULL,
    [EventSource] [int] NOT NULL,
    [DataField] [nvarchar](max) NULL,
    [ExtensionData] [xml] NULL,
    [CreateDate] [datetime] NOT NULL,
    [ParentEventID] [bigint] NULL,
    [Deleted] [bit] NOT NULL,
 CONSTRAINT [PK_EventLog] PRIMARY KEY CLUSTERED 
(
    [EventID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, 
 ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]


-- Non-clustered "covering index"
CREATE NONCLUSTERED INDEX [IX_EventLogNew_Cover1] ON [dbo].[EventLogNew] 
(
    [UserID] ASC,
    [Deleted] ASC,
    [CreateDate] ASC
)
INCLUDE ( [EventID],
[GroupID],
[PostID],
[FeedID],
[EventChange],
[EventObject],
[EventType],
[EventSource]) WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, 
SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, 
ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
GO


-- Return IO Statistics
SET STATISTICS IO ON;

-- Clear out the procedure cache for instance    
DBCC FREEPROCCACHE;
    
-- Don't do this on a production server!
DBCC DROPCLEANBUFFERS;

-- Run four identical stored procedures
-- against identical tables    (except for index changes)
PRINT 'Query 1'; -- Clustered Index Scan
EXEC dbo.GetEventLogByUserID 137926;
PRINT 'Query 2'; -- Clustered Index Scan (compressed index)
EXEC dbo.GetEventLogCompressedByUserID 137926;
PRINT 'Query 3'; -- Index seek
EXEC dbo.GetEventLogNewByUserID 137926;
PRINT 'Query 4'; -- Index seek (compressed index)
EXEC dbo.GetEventLogNewCompressedByUserID 137926;

-- Statistics IO output
--Query 1 Clustered index scan against EventLog table
--Table 'EventLog'. Scan count 3, logical reads 23941, physical reads 176, 
--read-ahead reads 23858, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

--Query 2 Clustered index scan against EventLogCompressed table
--Table 'EventLogCompressed'. Scan count 3, logical reads 5545, physical reads 45, 
--read-ahead reads 5509, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

--Query 3 Index seek against EventLogNew
--Table 'EventLogNew'. Scan count 1, logical reads 3, physical reads 3, 
--read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

--Query 4 Index seek against EventLogNewCompressed
--Table 'EventLogNewCompressed'. Scan count 1, logical reads 3, physical reads 2, 
--read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.


-- Breaks down buffers used by current database 
-- by object (table, index) in the buffer cache
SELECT OBJECT_NAME(p.[object_id]) AS [ObjectName], 
p.index_id, COUNT(*)/128 AS [buffer size(MB)],  
COUNT(*) AS [buffer_count] 
FROM sys.allocation_units AS a
INNER JOIN sys.dm_os_buffer_descriptors AS b
ON a.allocation_unit_id = b.allocation_unit_id
INNER JOIN sys.partitions AS p
ON a.container_id = p.hobt_id
WHERE b.database_id = DB_ID()
AND p.[object_id] > 100
GROUP BY p.[object_id], p.index_id
ORDER BY buffer_count DESC;

-- Buffer space results
--ObjectName                index_id    buffer size(MB)    buffer_count
--EventLog                           1            186                23855
--EventLogCompressed        1             43                 5508
--EventLogNew                    5              0                   24
--EventLogNewCompressed  2              0                   16 

SQL Server 2008: Backup Compression CPU Cost


Back in September last year I blogged about the native Backup Compression that is in SQL Server 2008 (see
 here) and promised to blog more when CTP-5 came out and I ran some tests. Well, it’s here and I have so here’s some data for you.


I expanded the AdventureWorks database to be 322Mb (random size, but big enough to get a decent sized run-time on my server). I used System Monitor to capture %user-mode CPU time, plus backup/restore throughput for a compressed and uncompressed backup operation, and then restores.


1) For the uncompressed backup the average CPU was 5% (the green line at the bottom), the run-time was 39.5s, and, of course, it took 322Mb to store the backup.


2) For the compressed backup the average CPU was way higher at 25%, BUT the run-time was 21.6s (a 45% improvement), and the backup was stored in 76.7MB (a 4.2x compression ratio). Very cool.


3) For the restore of the uncompressed backup the average CPU was 8%, and the run-time was 71.0s.


4) For the restore of the compressed backup the average CPU was 14.5%, and the run-time was 36s (a 50% improvement).

So – to summarize, turning on compression means more CPU and smaller run-times – just what was expected. Note that if you try this on your database you will see different results – the compression ratio and CPU usage is entirely dependent on the data being compressed.

SQL Server 2008: Backup Compression- Part 1

First up is Backup Compression. This has been one of most heavily requested features since before I joined the SQL team back in 1999 – for me it’s really cool that it’s in this coming release.
It’s widely recognized that database sizes are growing significantly, and that the growth is being forced by many factors, such as companies requiring more data to be available online for longer (e.g. to comply with government regulations) or an increasing amount of data being digitized for storage. Although the per/GB cost of storage is falling, the actual cost of keeping a database available is increasing. Here’s why:
  • Prior to SQL Server 2008, a backup of a VLDB (or any sized database really) takes, on average, the same amount of disk space as the data in the database itself (remember that database files can be sized much larger than the amount of data they hold). A prudent backup strategy is to have multiple backups online, and multiple redundant copies of each backup. Thus, the amount of disk space required increases rather quickly – more than offsetting the drop in storage costs.
  • Keeping safe copies of all database backups means they have to be physically copied to a separate location. Increasing database sizes translates into more bandwidth required to copy larger database backups (or lower bandwidth links get saturated for longer and longer).
  • If you have a well designed storage strategy, then you’re probably using RAID – so the more backups you need to store, and the higher level of RAID you use, the more drives you need to store all these backup copies.
In addition to costs, the elapsed times of backup and restore operations need to be considered. Backup and restore operations are essentially IO-bound. For a backup in SQL Server 2005 and before, the whole database has to be read and then written to a new location – with the total IO size of the writes to the backup equalling that of the reads from the database. The same is true for a restore, with reads from the backup equalling writes to the restored database. (As an aside, a restore operation also has the added CPU cost of having to examine each page as it’s read from the backup to figure out where it should be written to in the restored database – this usually makes a restore operation take 10-20% more elapsed time than a backup.)
The perfect solution to these problems, at the expense of sometimes-considerable CPU time, is to compress the database as it’s written into the backup. This reduces storage space per backup, required network bandwidth per backup copy, and elapsed time for backup and restore operations. The last point – reducing elapsed time for restore operations – is especially important in disaster recovery situations, where anything that can help reduce database downtime is a good thing.
A less ideal solution I’ve seen is to take a regular backup and then compress it after-the-fact before copying to other locations. While this is a reasonable solution, it requires more disk space than compressing the database as its backed up, and it’s a more complicated procedure. It also increases the time to take the backup, as the compression is done in a seperate step.
An alternative to compressing the backup at all is to make the backup location a compressed NTFS directory. While this achieves the compression goal, it doesn’t permanently compress the backup so doesn’t reduce the network bandwidth required to copy the backup or the space needed to archive the backup to tape.
Up until SQL Server 2008, the only solutions for compression-during-backup have come from third-party software vendors. Although these solutions do the job of aleviating the problems I’ve described above, there are two major roadblocks to their adoption that I’ve heard from SQL Server customers:
  1. You need to buy another software license as well as SQL Server – this can be pretty expensive for a large number of SQL Server installations. Management is also a headache, to ensure that all sites that may need to decompress the backup have the correct software installed.
  2. Some IT shops are Microsoft-only, which precludes the use of any software not supplied by Microsoft.
In SQL Server 2008, Microsoft will include a long-awaited and much-asked-for backup compression solution – eliminating the two roadblocks above. This is a fantastic first step improving backup/restore functionality – hopefully in the release after SQL Server 2008 we’ll see further innovations that will allow encrypted backups, table-level restore, and easier validation of the database stored in a backup.
Some points to note:
  • Adhering to the principal-of-least-surprise, backup compression will be off by default, with very simple syntax to turn it on – directly with T-SQL or through the tools.
  • The compression algorithm used is proprietary to Microsoft and has yielded similar compression ratios to well-known third-party products.
  • During Microsoft’s in-house testing on real -world customer databases, average compression ratios of 5:1 have been observed.
  • A backup set will not be able to contain both compressed and uncompressed backups.
  • None of the existing functionality will be altered by compression – e.g. the operation of WITH CHECKSUM or WITH CONTINUE_AFTER_ERROR (see here for more info on those options).
  • A restore operation will be able to tell automatically whether a given backup is compressed or not and just do the right thing.
Once backup compression is available in a public CTP, I’ll blog some sample scripts and report on things like:
  • the compression ratios, elapsed time differences, and CPU usages for a few sample databases
  • the varoius configuration options available
  • any differences in the MSDB backup history tables or the output from RESTORE HEADERONLY/LABELONLY
In summary, backup compression is a very exciting feature and should enable many customers to save money, either on additional software licenses or on storage/network costs, and time, especially in all-important disaster recovery situations.

SQL Server 2008: Backup Compression - Part 2