Обсуждение: Hstore::TooBig ?
I am running a somewhat unfamiliar Ruby automation results app using ActiveRecord to manage the postgres 9.0 backend.
Duringour automation runs we sometimes get bursts of HTTP 500 errors coming back at us, and the Ruby app log shows an
Hstore::TooBigerror:
Hstore::TooBig (Hstore::TooBig):
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:3045:in `attributes_with_quotes'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:3036:in `each'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:3036:in `attributes_with_quotes'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:2934:in `update_without_lock'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/locking/optimistic.rb:81:in `update_without_dirty'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/dirty.rb:146:in `update_without_timestamps'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/timestamp.rb:64:in `update_without_callbacks'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/callbacks.rb:282:in `update'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:2927:in
`create_or_update_without_callbacks'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/callbacks.rb:250:in `create_or_update'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:2594:in `save_without_validation!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/validations.rb:1100:in `save_without_dirty!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/dirty.rb:87:in `save_without_transactions!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:200:in `save!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/connection_adapters/abstract/database_statements.rb:136:in
`transaction'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:182:in `transaction'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:200:in `save!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:208:in
`rollback_active_record_state!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:200:in `save!'
app/helpers/logger_helper.rb:72:in `update_testrun'
app/models/testrun.rb:267:in `transaction_with_isolation_level'
...
I don't quite know where this error is coming from. The ActiveRecord source doesn't seem to have it, and I'm not
familiarenough with Rails or ActiveRecord to track definitively whether the failing function is actually performing a
DBoperation. Moreso I cannot seem to find any mentions (post 9.0 at least) of Hstore size limitations, either key/val
pairs(which 8.x docs said was 65k) or the hstore field in its entirety.
So to piece out the questions,
- is there a total size limitation of the hstore field? or is it theoretically large enough (1GB) that it really
shouldn'tmatter?
- is there a string size limitation of each key/val in 9.x, or is it still 65k from 8.x?
- is Hstore::TooBig an error that is likely coming from postgres? or should I continue poking at ActiveRecord or other
places?
thanks,
~ john
On Oct 31, 2013, at 3:46 PM, john gale <john@smadness.com> wrote:
I don't quite know where this error is coming from. The ActiveRecord source doesn't seem to have it, and I'm not familiar enough with Rails or ActiveRecord to track definitively whether the failing function is actually performing a DB operation. Moreso I cannot seem to find any mentions (post 9.0 at least) of Hstore size limitations, either key/val pairs (which 8.x docs said was 65k) or the hstore field in its entirety.
I should note there is no error thrown into my postgres log at all.
~ john
On 10/31/2013 03:46 PM, john gale wrote: > > I am running a somewhat unfamiliar Ruby automation results app using ActiveRecord to manage the postgres 9.0 backend. During our automation runs we sometimes get bursts of HTTP 500 errors coming back at us, and the Ruby app log shows an Hstore::TooBigerror: > > Hstore::TooBig (Hstore::TooBig): > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:3045:in `attributes_with_quotes' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:3036:in `each' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:3036:in `attributes_with_quotes' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:2934:in `update_without_lock' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/locking/optimistic.rb:81:in `update_without_dirty' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/dirty.rb:146:in `update_without_timestamps' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/timestamp.rb:64:in `update_without_callbacks' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/callbacks.rb:282:in `update' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:2927:in `create_or_update_without_callbacks' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/callbacks.rb:250:in `create_or_update' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:2594:in `save_without_validation!' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/validations.rb:1100:in `save_without_dirty!' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/dirty.rb:87:in `save_without_transactions!' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:200:in `save!' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/connection_adapters/abstract/database_statements.rb:136:in `transaction' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:182:in `transaction' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:200:in `save!' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:208:in `rollback_active_record_state!' > vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:200:in `save!' > app/helpers/logger_helper.rb:72:in `update_testrun' > app/models/testrun.rb:267:in `transaction_with_isolation_level' > ... > > I don't quite know where this error is coming from. The ActiveRecord source doesn't seem to have it, and I'm not familiarenough with Rails or ActiveRecord to track definitively whether the failing function is actually performing a DBoperation. Moreso I cannot seem to find any mentions (post 9.0 at least) of Hstore size limitations, either key/val pairs(which 8.x docs said was 65k) or the hstore field in its entirety. > > So to piece out the questions, > - is there a total size limitation of the hstore field? or is it theoretically large enough (1GB) that it really shouldn'tmatter? > - is there a string size limitation of each key/val in 9.x, or is it still 65k from 8.x? > - is Hstore::TooBig an error that is likely coming from postgres? or should I continue poking at ActiveRecord or otherplaces? Two things I can think of. 1) Grep the above gems for the Hstore::TooBig message 2) Ask on a Rails list, in light of your second post where you say the error does not touch the Postgres log. Would seem to indicate a Ruby/Rails issue. > > thanks, > > ~ john > > -- Adrian Klaver adrian.klaver@gmail.com
On Oct 31, 2013, at 4:44 PM, Adrian Klaver <adrian.klaver@gmail.com> wrote:
So to piece out the questions,
- is there a total size limitation of the hstore field? or is it theoretically large enough (1GB) that it really shouldn't matter?
- is there a string size limitation of each key/val in 9.x, or is it still 65k from 8.x?
- is Hstore::TooBig an error that is likely coming from postgres? or should I continue poking at ActiveRecord or other places?
Two things I can think of.
1) Grep the above gems for the Hstore::TooBig message
2) Ask on a Rails list, in light of your second post where you say the error does not touch the Postgres log. Would seem to indicate a Ruby/Rails issue.
These were done in parallel, which is why I was asking the question on a postgres list.
The further questions still apply though; are there documented size limitations for hstore?
~ john
On Oct 31, 2013, at 4:44 PM, Adrian Klaver <adrian.klaver@gmail.com> wrote:
So to piece out the questions,
- is there a total size limitation of the hstore field? or is it theoretically large enough (1GB) that it really shouldn't matter?
- is there a string size limitation of each key/val in 9.x, or is it still 65k from 8.x?
- is Hstore::TooBig an error that is likely coming from postgres? or should I continue poking at ActiveRecord or other places?
Two things I can think of.
1) Grep the above gems for the Hstore::TooBig message
2) Ask on a Rails list, in light of your second post where you say the error does not touch the Postgres log. Would seem to indicate a Ruby/Rails issue.
These were done in parallel, which is why I was asking the question on a postgres list.
The further questions still apply though; are there documented size limitations for hstore?
~ john
On 10/31/2013 5:11 PM, john gale wrote: > The further questions still apply though; are there documented size > limitations for hstore? afaik only the practical limits of how big you want your tables to get, and the 1GB limit of the underlying text representation. having many 1000s of keys in a single hstore would likely make for some painful performance, and nasty memory requirements (returning all keynames, for instance, so you could enumerate them, ouch) -- john r pierce 37N 122W somewhere on the middle of the left coast