加入收藏 | 设为首页 | 会员中心 | 我要投稿 李大同 (https://www.lidatong.com.cn/)- 科技、建站、经验、云计算、5G、大数据,站长网!
当前位置: 首页 > 大数据 > 正文

perl – 由于缓冲导致死锁.它是如何工作的?

发布时间:2020-12-15 23:37:12 所属栏目:大数据 来源:网络整理
导读:以下问题是对 this答案中关于死锁的评论的回应.我很好奇死锁是如何产生的,所以我创建了一个测试程序:有一个父进程将大量数据写入子进程STDIN,而子进程也将大量数据写入父进程的读取器句柄.事实证明,如果数据大小超过80K(Ubuntu 16.04),将会出现死锁: paren
以下问题是对 this答案中关于死锁的评论的回应.我很好奇死锁是如何产生的,所以我创建了一个测试程序:有一个父进程将大量数据写入子进程STDIN,而子进程也将大量数据写入父进程的读取器句柄.事实证明,如果数据大小超过80K(Ubuntu 16.04),将会出现死锁:

parent.pl:

use feature qw(say);
use strict;
use warnings;

use IPC::Open2;

my $test_size = 80_000; # How many bytes to write?
my $pid = open2( my $reader,my $writer,'child.pl' );

my $long_string = '0123456789' x ($test_size / 10);
printf "Parent: writing long string ( length: %d )n",length $long_string;
print $writer $long_string;
close $writer;
say "Parent: Trying to read childs ouput..";
my $output = do { local $/; <$reader> };
printf "Parent: Got output with length %d..n",length $output;
close $reader;
say "Parent: Reaping child..";
waitpid( $pid,0 );
my $child_exit_status = $? >> 8;
say "Parent: Child exited with status: $child_exit_status";

child.pl:

use feature qw(say);
use strict;
use warnings;

my $test_size = 80_000; # How many bytes to write?
my $child_log_filename = 'childlog.txt';

open ( my $log,'>',$child_log_filename ) or die "Could not create log file: $!";

say $log "Child is running..";

my $long_string = '0123456789' x ($test_size / 10);
say $log "Length of output string: " . length $long_string;
say $long_string;

my $input = do { local $/; <STDIN> };
say $log "Length of input string: " . length $input;
exit 2;

为什么这个程序会死锁?

解决方法

死锁的原因似乎是管道在写入时被填满,没有人从中读取.根据 pipe(7):

If a process attempts to write to a full pipe (see below),then
write(2) blocks until sufficient data has been read from the pipe to
allow the write to complete.

A pipe has a limited capacity. If the pipe is full,then a write(2)
will block or fail,depending on whether the O_NONBLOCK flag is set.
Different implementations have different limits for the pipe capacity.
Applications should not rely on a particular capacity: an application
should be designed so that a reading process consumes data as soon as
it is available,so that a writing process does not remain blocked.

In Linux versions before 2.6.11,the capacity of a pipe was the same
as the system page size (e.g.,4096 bytes on i386). Since Linux
2.6.11,the pipe capacity is 65536 bytes. Since Linux 2.6.35,the default pipe capacity is 65536 bytes

因此,父级将一个长字符串写入子级的STDIN管道,管道被填满,这会导致父级阻塞.与此同时,孩子正在向父母读者管道写一个长串,这也会被填满并且孩子会阻塞.因此,孩子正在等待父母从其阅读器管道中读取,而父母正在等待孩子从其STDIN管道中读取.一个所谓的僵局.

我试图通过使用select,fcntl,sysread和syswrite来解决这个问题.如果我们尝试写入超过管道容量的syswrite将阻塞,我使用fcntl使编写器句柄非阻塞.在这种情况下,syswrite将尽可能多地写入管道,然后立即返回它实际写入的字节数.

注意:只需要更改parent.pl(这也是需要的,因为我们不应该假设可以访问子源).这是
修改后的parent.pl将防止死锁:

use feature qw(say);
use strict;
use warnings;

use Errno qw( EAGAIN );
use Fcntl;
use IO::Select;
use IPC::Open2;

use constant READ_BUF_SIZE => 8192;
use constant WRITE_BUF_SIZE => 8192;

my $test_size = 80_000; # How many bytes to write?
my $pid = open2( my $reader,'child.pl' );

make_filehandle_non_blocking( $writer );

my $long_string = '0123456789' x ($test_size / 10);
printf "Parent: writing long string ( length: %d )n",length $long_string;
my $sel_writers = IO::Select->new( $writer );
my $sel_readers = IO::Select->new( $reader );
my $read_offset = 0;
my $write_offset = 0;
my $child_output = '';
while (1) {
    last if $sel_readers->count() == 0 && $sel_writers->count() == 0;
    my @sel_result = IO::Select::select( $sel_readers,$sel_writers,undef );
    my @read_ready = @{ $sel_result[0] };
    my @write_ready = @{ $sel_result[1] };
    if ( @write_ready ) {
        my $bytes_written = syswrite $writer,$long_string,WRITE_BUF_SIZE,$write_offset;
        if ( !defined $bytes_written ) {
            die "syswrite failed: $!" if $! != EAGAIN;
            $bytes_written = 0;
        }
        $write_offset += $bytes_written;
        if ( $write_offset >= length $long_string ) {
            $sel_writers->remove( $writer );
            close $writer;
        }
    }
    if ( @read_ready ) {
        my $bytes_read = sysread $reader,$child_output,READ_BUF_SIZE,$read_offset;
        if ( !defined $bytes_read ) {
            die "sysread failed: $!" if $! != EAGAIN;
            $bytes_read = 0;
        }
        elsif ( $bytes_read == 0 ) {
            $sel_readers->remove( $reader );
            close $reader;
        }
        $read_offset += $bytes_read;
    }
}
printf "Parent: Got output with length %d..n",length $child_output;
say "Parent: Reaping child..";
waitpid( $pid,0 );
my $child_exit_status = $? >> 8;
say "Parent: Child exited with status: $child_exit_status";

sub make_filehandle_non_blocking {
    my ( $fh ) = @_;

    my $flags = fcntl $fh,F_GETFL,0
      or die "Couldn't get flags for file handle : $!n";
    fcntl $fh,F_SETFL,$flags | O_NONBLOCK
      or die "Couldn't set flags for file handle: $!n";

}

(编辑:李大同)

【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容!

    推荐文章
      热点阅读