Archive for June 2015

NTFS File/Folder and Path Limits

ntfs

What is a file system?

A file system is a part of the operating system that determines how files are named, stored, and organized on a volume. A file system manages files and folders, and the information needed to locate and access these items by local and remote users. NTFS, short for New Technology File System, is a file system that was introduced by Microsoft in 1993 with Windows NT 3.1.

Benefits of NTFS

  • Increasing reliability

NTFS uses its log file and checkpoint information to restore the consistency of the file system when the computer is restarted in the event of a system failure. In the event of a bad-sector error, NTFS dynamically remaps the cluster containing the bad sector and allocates a new cluster for the data, as well as marking the cluster as bad and no longer using it. For example, by formatting a POP3 mail server with NTFS, the mail store can offer logging and recovery. In the event of a server crash, NTFS can recover data by replaying its log files.

  • Increasing security

NTFS allows you to set permissions on a file or folder, and specify the groups and users whose access you want to restrict or allow, and then select the type of access. NTFS also supports the Encrypting File System (EFS) technology used to store encrypted files on NTFS volumes. Any intruder who tries to access your encrypted files is prevented from doing so, even if that intruder has physical access to the computer. For example, a POP3 mail server, when formatted with an NTFS file system, provides increased security for the mail store, security that would not be available should the server be formatted with the FAT file system.

  • Supporting large volumes

NTFS allows you to create an NTFS volumes as per below

  1. Up to 16 terabytes using the default cluster size (4 KB) for large volumes.
  2. Up to 256 terabytes using the maximum cluster size of 64 KB.
  3. NTFS also supports larger files and more files per volume than FAT File Systems.

Limited space on a volume

If your organization has limited space on a volume, NTFS provides support for increasing storage on a server with limited disk space.

  1. Disk quotas allow you to track and control user disk space usage for NTFS volumes.
  2. NTFS supports compression as well as adding unallocated space from the same disk or from another disk to increase the size of an NTFS volume.
  3. Mounted volumes allow you to mount a volume at any empty folder on a local NTFS volume if you run out of drive letters or need to create additional space that is accessible from an existing folder.

Using features available only in NTFS

NTFS has a number of features that are not available if you are using a FAT file system. These include:

  1. Distributed link tracking. Maintains the integrity of shortcuts and OLE links. You can rename source files, move them to NTFS volumes on different computers within a Windows Server 2003 or Windows 2000 domain, or change the computer name or folder name that stores the target without breaking the shortcut or OLE links.
  2. Sparse files. Large, consecutive areas of zeros. NTFS manages sparse files by tracking the starting and ending point of the sparse file, as well as its useful (non-zero) data. The unused space in a sparse file is made available as free space.
  3. NTFS change journal. Provides a persistent log of changes made to files on a volume. NTFS maintains the change journal by tracking information about added, deleted, and modified files for each volume.
  4. Hard links. NTFS-based links to a file on an NTFS volume. By creating hard links, you can have a single file in multiple folders without duplicating the file. You can also create multiple hard links for a file in a folder if you use different file names for the hard links. Because all of the hard links reference the same file, applications can open any of the hard links and modify the file.
  • Volume Shadow Copy Service

Service that provides an infrastructure for creating highly accurate, point-in-time shadow copies. These copies of a single volume or multiple volumes can be made without affecting the performance of a production server. The Volume Shadow Copy Service can produce accurate shadow copies by coordinating with business applications, backup applications, and storage hardware.

  • Distributed File System (DFS).

Strategic storage management solution in Windows Server 2003 that enables you to group shared folders located on different servers logically by transparently connecting them to one or more hierarchical namespaces.

  • File System Replication (FRS)

Technology that replicates files and folders stored in the SYSVOL shared folder on domain controllers and Distributed File System (DFS) shared folders. When FRS detects that a change has been made to a file or folder within a replicated shared folder, FRS replicates the updated file or folder to other servers

FAT32 and NTFS Limits

FAT32:

  • Maximum disk size: 2 terabytes
  • Maximum file size: 4 gigabytes
  • Maximum number of files on disk: 268,435,437
  • Maximum number of files in a single folder: 65,534

NTFS:

  • Maximum disk size: 256 terabytes
  • Maximum file size: 256 terabytes
  • Maximum number of files on disk: 4,294,967,295
  • Maximum number of files in a single folder: 4,294,967,295

File Path Lengths

In the Windows API, the maximum length for a path is MAX_PATH, which is defined as 260 characters. A local path is structured in the following order: drive letter, colon, backslash, name components separated by backslashes, and a terminating null character. For example, the maximum path on drive D is “D:\some 256-character path string” where “” represents the invisible terminating null character for the current system codepage. (The characters < > are used here for visual clarity and cannot be part of a valid path string.)

The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters. This type of path is composed of components separated by backslashes, each up to the value returned in the lpMaximumComponentLength parameter of the GetVolumeInformation function (this value is commonly 255 characters). To specify an extended-length path, use the “\\?\” prefix. For example, “\\?\D:\very long path“.

Long Path Tool

There is a brilliant piece of software called Long Path Tool. This can scan a directory or folder and tell you which paths are over the 256 character limit

http://longpathtool.com/

LongPathTool

GetFolderSize

This is another piece of free software which can tell you folder and file sizes for a directory and folders

http://www.getfoldersize.com/en_download.htm#info

GetFolderSize

Useful Microsoft Link for detailed NTFS information

https://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx