postgresql – 如何在没有文件或内存的情况下将IO :: Handle子类
我有一个访问PostgreSQL数据库的应用程序,需要根据一些需要的处理从中读取一些大的二进制数据.这可能是数百MB甚至数GB的数据.请不要讨论使用文件系统等等,它就像现在这样.
该数据只是各种类型的文件,例如它可能是Zip容器或其他类型的存档.一些需要的处理是列出Zip的内容,甚至可以提取一些成员进行进一步处理,也可以散列存储的数据……最后,数据被多次读取,但只写入一次以存储它. 我使用的所有Perl库都可以使用文件句柄,一些使用IO :: Handle,另一些使用IO :: String或IO :: Scalar,其他一些只使用低级文件句柄.所以我所做的是创建一个IO :: Handle和IO :: Seekable的子类,它就像DBD :: Pg周围相应方法的包装器一样.在CTOR中,我创建了一个与数据库的连接,打开一些提供的LOID用于读取并存储Postgres在实例中提供的句柄.然后我自己的句柄对象被转发给能够使用这种文件句柄的人,并且可以直接在Postgres提供的blob中读取和搜索. 问题是在IO :: Handle上使用低级文件句柄或低级文件句柄操作的库.摘要:: MD5似乎是一个,Archive :: Zip另一个. Digest :: MD5呱呱叫,告诉我没有提供句柄,另一方面,Archive :: Zip尝试从我的创建一个新的,自己的句柄,调用IO :: Handle :: fdopen并在我的情况下失败. sub fdopen { @_ == 3 or croak 'usage: $io->fdopen(FD,MODE)'; my ($io,$fd,$mode) = @_; local(*GLOB); if (ref($fd) && "".$fd =~ /GLOB(/o) { # It's a glob reference; Alias it as we cannot get name of anon GLOBs my $n = qualify(*GLOB); *GLOB = *{*$fd}; $fd = $n; } elsif ($fd =~ m#^d+$#) { # It's an FD number; prefix with "=". $fd = "=$fd"; } open($io,_open_mode_string($mode) . '&' . $fd) ? $io : undef; } 我想问题是句柄的低级副本,它删除了我自己的实例,因此没有实例拥有我的数据库连接和所有这些东西. 那么,在我的情况下,甚至可以提供一些IO :: Handle,它可以在任何需要低级文件句柄的地方成功使用吗? 我的意思是,我没有真正的文件句柄,我只有一个对象,方法调用被包装到相应的Postgres方法,需要数据库句柄等.所有这些数据都需要存储在某个地方,需要完成包装等. 我尝试做其他人正在做的事情,比如IO :: String,例如另外使用tie.但最终用例是不同的,因为Perl能够自己创建一个真正的低级文件句柄到一些内部内存.在我的情况下根本不支持的东西.我需要保持我的实例,因为只知道数据库的句柄等. 通过调用方法读取来使用我的句柄,如IO :: Handle,这样的工作就像预期的那样,但是我想更进一步,并且与那些不期望在IO :: Handle对象上工作的人更加兼容.很像IO :: String或File :: Temp可以用作低级文件句柄. package ReadingHandle; use strict; use warnings; use 5.10.1; use base 'IO::Handle','IO::Seekable'; use Carp (); sub new { my $invocant = shift || Carp::croak('No invocant given.'); my $db = shift || Carp::croak('No database connection given.'); my $loid = shift // Carp::croak('No LOID given.'); my $dbHandle = $db->_getHandle(); my $self = $invocant->SUPER::new(); *$self->{'dbHandle'} = $dbHandle; *$self->{'loid'} = $loid; my $loidFd = $dbHandle->pg_lo_open($loid,$dbHandle->{pg_INV_READ}); *$self->{'loidFd'} = $loidFd; if (!defined($loidFd)) { Carp::croak("The provided LOID couldn't be opened."); } return $self; } sub DESTROY { my $self = shift || Carp::croak('The method needs to be called with an instance.'); $self->close(); } sub _getDbHandle { my $self = shift || Carp::croak('The method needs to be called with an instance.'); return *$self->{'dbHandle'}; } sub _getLoid { my $self = shift || Carp::croak('The method needs to be called with an instance.'); return *$self->{'loid'}; } sub _getLoidFd { my $self = shift || Carp::croak('The method needs to be called with an instance.'); return *$self->{'loidFd'}; } sub binmode { my $self = shift || Carp::croak('The method needs to be called with an instance.'); return 1; } sub close { my $self = shift || Carp::croak('The method needs to be called with an instance.'); my $dbHandle = $self->_getDbHandle(); my $loidFd = $self->_getLoidFd(); return $dbHandle->pg_lo_close($loidFd); } sub opened { my $self = shift || Carp::croak('The method needs to be called with an instance.'); my $loidFd = $self->_getLoidFd(); return defined($loidFd) ? 1 : 0; } sub read { my $self = shift || Carp::croak('The method needs to be called with an instance.'); my $buffer =shift // Carp::croak('No buffer given.'); my $length = shift // Carp::croak('No amount of bytes to read given.'); my $offset = shift || 0; if ($offset > 0) { Carp::croak('Using an offset is not supported.'); } my $dbHandle = $self->_getDbHandle(); my $loidFd = $self->_getLoidFd(); return $dbHandle->pg_lo_read($loidFd,$buffer,$length); } sub seek { my $self = shift || Carp::croak('The method needs to be called with an instance.'); my $offset = shift // Carp::croak('No offset given.'); my $whence = shift // Carp::croak('No whence given.'); if ($offset < 0) { Carp::croak('Using a negative offset is not supported.'); } if ($whence != 0) { Carp::croak('Using a whence other than 0 is not supported.'); } my $dbHandle = $self->_getDbHandle(); my $loidFd = $self->_getLoidFd(); my $retVal = $dbHandle->pg_lo_lseek($loidFd,$offset,$whence); $retVal = defined($retVal) ? 1 : 0; return $retVal; } sub tell { my $self = shift || Carp::croak('The method needs to be called with an instance.'); my $dbHandle = $self->_getDbHandle(); my $loidFd = $self->_getLoidFd(); my $retVal = $dbHandle->pg_lo_lseek($loidFd); $retVal = defined($retVal) ? $retVal : -1; return $retVal; } 1; 解决方法
有一种解决方法,但它有点奇怪.如果我正确阅读您的代码和评论,您的要求基本上是三倍的:
>尽可能像普通文件句柄/ IO :: Handle对象一样工作,这使得它不是用户不可见的真实文件. 您可以使用PerlIO layers with 以下是一个可满足您需求的示例包.它有一些警告: >它根本不扩展您的代码或与DB交互;它只使用提供的行arrayref作为文件数据.如果这似乎适合您的用例,您应该使其适应数据库. 包: package TiedThing; use strict; use warnings; use parent "IO::File"; our @pushargs; sub new { my ( $class,$args ) = @_; # Build a glob to be used by the PerlIO methods. This does two things: # 1. Gets us a place to stick a shared hashref so PerlIO methods and user- # -defined object methods can manipulate the same data. They must use the # {args} glob field to do that; new fields written will . # 2. Unifies the ways of addressing that across custom functions and PerlIO # functions. We could just pass a hashref { args => $args } into PUSHED,but # then we'd have to remember "PerlIO functions receive a blessed hashref,# custom functions receive a blessed glob" which is lame. my $glob = Symbol::gensym(); *$glob->{args} = $args; local @pushargs = ($glob,$class); my $self = $class->SUPER::new(my $unused,"<:via($class)"); *$self->{args} = $args; return $self; } sub custom { my $self = shift; return *$self->{args}->{customvalue}; } sub PUSHED { return bless($pushargs[0],$pushargs[1]); } sub FILL { return shift(@{*$_[0]->{args}->{lines}}); } 1; 用法示例: my $object = TiedThing->new({ lines => [join("n",1..9,1..9)],customvalue => "custom!",}); say "can call custom method: " . $object->custom; say "raw read with <>: " . <$object>; my $buf; read($object,$buf,10); say "raw read with read(): " . $buf; undef $buf; $object->read($buf,10); say "OO read via IO::File::read (end): " . $buf; my $checksummer = Digest::MD5->new;; $checksummer->addfile($object); say "Md5 read: " . $checksummer->hexdigest; my $dupto = IO::Handle->new; # Doesn't break/return undef; still not usable without implementing # more state sharing inside the object. say "Can dup handle: " . $dupto->fdopen($object,"r"); my $archiver = Archive::Zip->new; # Dies,but long after the fdopen() call. Can be fixed by implementing more # PerlIO methods. $archiver->readFromFileHandle($object); (编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |