根据大小将文本文件拆分为较小的文件(Windows)

注意：无法严格执行输出文件大小限制.

示例：输入文件包含两个非常长的字符串,每个字符串1Mb.如果您尝试将此文件拆分为512KB块,则生成的文件将分别为1MB.

功能Split-FileByLine：

<#
.Synopsis
    Split text file(s) by lines,put into each output file as many complete lines of input as possible without exceeding size bytes.

.Description
    Split text file(s) by lines,put into each output file as many complete lines of input as possible without exceeding size bytes.
    Note,that output file size limit can't be strictly enforced. Example: input files contains two very long strings,1Mb each.
    If you try to split this file into the 512KB chunks,resulting files will be 1MB each.

    Splitted files will have orinignal file's name,followed by the "_part_" string and counter. Example:
    Original file: large.log
    Splitted files: large_part_0.log,large_part_1.log,large_part_2.log,etc.

.Parameter FileName
    Array of strings,mandatory. Filename(s) to split.

.Parameter OutPath
    String,mandatory. Folder,where splittedfiles will be stored. Will be created,if not exists.

.Parameter MaxFileSize
    Long,mandatory. Maximum output file size. When output file reaches this size,new file will be created.
    You can use PowerShell's multipliers: KB,MB,GB,TB,PB

.Parameter Encoding
    String. If not specified,script will use system's current ANSI code page to read the files.
    You can get other valid encodings for your system in PowerShell console like this:

    [System.Text.Encoding]::GetEncodings()

    Example:

    Unicode (UTF-7): utf-7
    Unicode (UTF-8): utf-8
    Western European (Windows): Windows-1252

.Example
    Split-FileByLine -FileName '.large.log' -OutPath '.splitted' -MaxFileSize 100MB -Verbose

    Split file "large.log" in current folder,write resulting files in subfolder "splitted",limit output file size to 100Mb,be verbose.

.Example
    Split-FileByLine -FileName '.large.log' -OutPath '.splitted' -MaxFileSize 100MB -Encoding 'utf-8'

    Split file "large.log" in current folder,use UTF-8 encoding.

.Example
    Split-FileByLine -FileName '.large_1.log','.large_2.log' -OutPath '.splitted' -MaxFileSize 999MB

    Split files "large_1.log" ".large_2.log" and  in current folder,limit output file size to 999MB.

.Example
    '.large_1.log','.large_2.log' | Split-FileByLine -FileName -OutPath '.splitted' -MaxFileSize 999MB

    Split files "large_1.log" ".large_2.log" and  in current folder,limit output file size to 999MB.

#>
function Split-FileByLine
{
    [CmdletBinding()]
    Param
    (
        [Parameter(Mandatory = $true,ValueFromPipeline = $true,ValueFromPipelineByPropertyName = $true)]
        [string[]]$FileName,[Parameter(ValueFromPipelineByPropertyName = $true)]
        [string]$OutPath = (Get-Location -PSProvider FileSystem).Path,[Parameter(Mandatory = $true,ValueFromPipelineByPropertyName = $true)]
        [long]$MaxFileSize,[Parameter(ValueFromPipelineByPropertyName = $true)]
        [string]$Encoding = 'Default'
    )

    Begin
    {
        # Scriptblocks for common tasks
        $DisposeInFile = {
            Write-Verbose 'Disposing StreamReader'
            $InFile.Close()
            $InFile.Dispose()
        }

        $DispoSEOutFile = {
            Write-Verbose 'Disposing StreamWriter'
            $OutFile.Flush()
            $OutFile.Close()
            $OutFile.Dispose()
        }

        $NewStreamWriter = {
            Write-Verbose 'Creating StreamWriter'
            $OutFileName = Join-Path -Path $OutPath -ChildPath (
                '{0}_part_{1}{2}' -f [System.IO.Path]::GetFileNameWithoutExtension($_),$Counter,[System.IO.Path]::GetExtension($_)
            )

            $OutFile = New-Object -TypeName System.IO.StreamWriter -ArgumentList (
                $OutFileName,$false,$FileEncoding
            ) -ErrorAction Stop
            $OutFile.AutoFlush = $true
            Write-Verbose "Writing new file: $OutFileName"
        }
    }

    Process
    {
        if($Encoding -eq 'Default')
        {
            # Set default encoding
            $FileEncoding = [System.Text.Encoding]::Default
        }
        else
        {
            # Try to set user-specified encoding
            try
            {
                $FileEncoding = [System.Text.Encoding]::GetEncoding($Encoding)
            }
            catch
            {
                throw "Not valid encoding: $Encoding"
            }
        }

        Write-Verbose "Input file: $FileName"
        Write-Verbose "Output folder: $OutPath"

        if(!(Test-Path -Path $OutPath -PathType Container)){
            Write-Verbose "Folder doesn't exist,creating: $OutPath"
            $null = New-Item -Path $OutPath -ItemType Directory -ErrorAction Stop
        }

        $FileName | ForEach-Object {
            # Open input file
            $InFile = New-Object -TypeName System.IO.StreamReader -ArgumentList (
                $_,$FileEncoding
            ) -ErrorAction Stop
            Write-Verbose "Current file: $_"

            $Counter = 0
            $OutFile = $null

            # Read lines from input file
            while(($line = $InFile.ReadLine()) -ne $null)
            {
                if($OutFile -eq $null)
                {
                    # No output file,create StreamWriter
                    . $NewStreamWriter
                }
                else
                {
                    if($OutFile.BaseStream.Length -ge $MaxFileSize)
                    {
                        # Output file reached size limit,closing
                        Write-Verbose "OutFile lenght: $($InFile.BaseStream.Length)"
                        . $DispoSEOutFile
                        $Counter++
                        . $NewStreamWriter
                    }
                }

                # Write line to the output file
                $OutFile.WriteLine($line)
            }

            Write-Verbose "Finished processing file: $_"
            # Close open files and cleanup objects
            . $DispoSEOutFile
            . $DisposeInFile
        }
    }
}

您可以在脚本中使用它,如下所示：

function Split-FileByLine
{
    # function body here
}

$InputFile = 'c:loglarge.log'
$OutputDir = 'c:log_split'

Split-FileByLine -FileName $InputFile -OutPath $OutputDir -MaxFileSize 999MB