Обсуждение: Re: [BUGS] no way in LargeObject API to detect short read?
Anyone able to fix this? > Paul M. Aoki (aoki@acm.org) reports a bug with a severity of 3 > The lower the number the more severe it is. > > Short Description no way in LargeObject API to detect short > read? > > Long Description > org.postgresql.largeobject.LargeObject.read(byte[],int,int) > returns void instead of int. there's therefore no reliable, > non-exceptional way to detect short reads. > > the natural way to write blocked-read loops would be to assume > that read(byte[],int,int) returned the number of bytes read or > (e.g.) -1 on EOF. > > Sample Code > > > No file was uploaded with this report > > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
Bruce Momjian <pgman@candle.pha.pa.us> writes: > Anyone able to fix this? here's a hack we've been using in-house (written by Jun Gabayan, <jgabayan@pahv.xerox.com>). you may not like the style but it's a stab at a solution. -- Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road aoki@acm.org / Computer Science Laboratory / Palo Alto, CA 94304-1314 Index: LargeObject.java =================================================================== RCS file: /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObject.java,v retrieving revision 1.1 retrieving revision 1.3 diff -r1.1 -r1.3 64c64,67 < --- > > private int pos = 0; //current position > private int size = 0; > 85a89,90 > pos = tell(); > size = size(); 102a108 > if(fd == 0) return; 105a112 > fd = 0; 118a126,132 > // calculate available data to read to avoid reading pass the end > // to avoid an exception > pos = tell(); > int avail = size - pos; > if(avail == 0) return null; > if(avail < len) len = avail; > try { 123c137,141 < --- > }catch(SQLException se) { > System.out.println("***LargeObject.read: Caught SQLException: " + se.getMessage()); > return null; > } > 157c175 < public void read(byte buf[],int off,int len) throws SQLException --- > public int read(byte buf[],int off,int len) throws SQLException 159c177,180 < System.arraycopy(read(len),0,buf,off,len); --- > byte mybuf[] = read(len); > int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for end of data > if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz); > return sz;
Hmmm, what's the performance issues with this? Is there going to be a problem with very large LargeObject's? Quoting "Paul M. Aoki" <aoki@acm.org>: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > Anyone able to fix this? > > here's a hack we've been using in-house (written by Jun Gabayan, > <jgabayan@pahv.xerox.com>). > > you may not like the style but it's a stab at a solution. > -- > Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road > aoki@acm.org / Computer Science Laboratory / Palo Alto, CA > 94304-1314 > > Index: LargeObject.java > =================================================================== > RCS file: > /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObje ct.java,v > retrieving revision 1.1 > retrieving revision 1.3 > diff -r1.1 -r1.3 > 64c64,67 > < > --- > > > > private int pos = 0; //current position > > private int size = 0; > > > 85a89,90 > > pos = tell(); > > size = size(); > 102a108 > > if(fd == 0) return; > 105a112 > > fd = 0; > 118a126,132 > > // calculate available data to read to avoid reading pass the > end > > // to avoid an exception > > pos = tell(); > > int avail = size - pos; > > if(avail == 0) return null; > > if(avail < len) len = avail; > > try { > 123c137,141 > < > --- > > }catch(SQLException se) { > > System.out.println("***LargeObject.read: Caught > SQLException: " + se.getMessage()); > > return null; > > } > > > 157c175 > < public void read(byte buf[],int off,int len) throws SQLException > --- > > public int read(byte buf[],int off,int len) throws SQLException > 159c177,180 > < System.arraycopy(read(len),0,buf,off,len); > --- > > byte mybuf[] = read(len); > > int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for > end of data > > if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz); > > return sz; > -- Peter Mount peter@retep.org.uk PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/ RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/
Peter, Given the current implementation of size(), I don't think this is a good solution to the problem at hand. Since size() is an expensive call (especially on large files), using it in this way wouldn't be a performant solution. Using size() also requires additional round trips to the database to get this information. There needs to be a better solution that doesn't require the additional overhead. thanks, --Barry Peter T Mount wrote: > > Hmmm, what's the performance issues with this? Is there going to be a problem > with very large LargeObject's? > > Quoting "Paul M. Aoki" <aoki@acm.org>: > > > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > > Anyone able to fix this? > > > > here's a hack we've been using in-house (written by Jun Gabayan, > > <jgabayan@pahv.xerox.com>). > > > > you may not like the style but it's a stab at a solution. > > -- > > Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road > > aoki@acm.org / Computer Science Laboratory / Palo Alto, CA > > 94304-1314 > > > > Index: LargeObject.java > > =================================================================== > > RCS file: > > /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObje > ct.java,v > > retrieving revision 1.1 > > retrieving revision 1.3 > > diff -r1.1 -r1.3 > > 64c64,67 > > < > > --- > > > > > > private int pos = 0; //current position > > > private int size = 0; > > > > > 85a89,90 > > > pos = tell(); > > > size = size(); > > 102a108 > > > if(fd == 0) return; > > 105a112 > > > fd = 0; > > 118a126,132 > > > // calculate available data to read to avoid reading pass the > > end > > > // to avoid an exception > > > pos = tell(); > > > int avail = size - pos; > > > if(avail == 0) return null; > > > if(avail < len) len = avail; > > > try { > > 123c137,141 > > < > > --- > > > }catch(SQLException se) { > > > System.out.println("***LargeObject.read: Caught > > SQLException: " + se.getMessage()); > > > return null; > > > } > > > > > 157c175 > > < public void read(byte buf[],int off,int len) throws SQLException > > --- > > > public int read(byte buf[],int off,int len) throws SQLException > > 159c177,180 > > < System.arraycopy(read(len),0,buf,off,len); > > --- > > > byte mybuf[] = read(len); > > > int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for > > end of data > > > if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz); > > > return sz; > > > > -- > Peter Mount peter@retep.org.uk > PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/ > RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/
Peter T Mount <peter@retep.org.uk> writes: > Hmmm, what's the performance issues with this? Is there going to be a problem > with very large LargeObject's? you could probably be smarter about caching previous tell() state, if that's what you mean. jun's hack doesn't actually add any extra buffers or copies. -- Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road aoki@acm.org / Computer Science Laboratory / Palo Alto, CA 94304-1314
At 15:37 26/01/01 -0800, Barry Lind wrote: >Peter, > >Given the current implementation of size(), I don't think this is a good >solution to the problem at hand. Since size() is an expensive call >(especially on large files), using it in this way wouldn't be a >performant solution. Using size() also requires additional round trips >to the database to get this information. There needs to be a better >solution that doesn't require the additional overhead. That's exactly what I was thinking of. Most of the largeobject API is based on libpq hence where this came from. I don't think there is anything available elsewhere for getting the size. Yes, there needs to be a better way. Peter >thanks, >--Barry > >Peter T Mount wrote: > > > > Hmmm, what's the performance issues with this? Is there going to be a > problem > > with very large LargeObject's? > > > > Quoting "Paul M. Aoki" <aoki@acm.org>: > > > > > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > > > Anyone able to fix this? > > > > > > here's a hack we've been using in-house (written by Jun Gabayan, > > > <jgabayan@pahv.xerox.com>). > > > > > > you may not like the style but it's a stab at a solution. > > > -- > > > Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road > > > aoki@acm.org / Computer Science Laboratory / Palo Alto, CA > > > 94304-1314 > > > > > > Index: LargeObject.java > > > =================================================================== > > > RCS file: > > > > /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObje > > ct.java,v > > > retrieving revision 1.1 > > > retrieving revision 1.3 > > > diff -r1.1 -r1.3 > > > 64c64,67 > > > < > > > --- > > > > > > > > private int pos = 0; //current position > > > > private int size = 0; > > > > > > > 85a89,90 > > > > pos = tell(); > > > > size = size(); > > > 102a108 > > > > if(fd == 0) return; > > > 105a112 > > > > fd = 0; > > > 118a126,132 > > > > // calculate available data to read to avoid reading pass the > > > end > > > > // to avoid an exception > > > > pos = tell(); > > > > int avail = size - pos; > > > > if(avail == 0) return null; > > > > if(avail < len) len = avail; > > > > try { > > > 123c137,141 > > > < > > > --- > > > > }catch(SQLException se) { > > > > System.out.println("***LargeObject.read: Caught > > > SQLException: " + se.getMessage()); > > > > return null; > > > > } > > > > > > > 157c175 > > > < public void read(byte buf[],int off,int len) throws SQLException > > > --- > > > > public int read(byte buf[],int off,int len) throws SQLException > > > 159c177,180 > > > < System.arraycopy(read(len),0,buf,off,len); > > > --- > > > > byte mybuf[] = read(len); > > > > int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for > > > end of data > > > > if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz); > > > > return sz; > > > > > > > -- > > Peter Mount peter@retep.org.uk > > PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/ > > RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/