Bug #523

rroongaでのUInt32 VectorへのArrayオブジェクトの代入でSegmentation fault

Added by Tasuku SUENAGA almost 8 years ago. Updated about 6 years ago.

Status:完了Start date:09/17/2010
Priority:NormalDue date:
Assignee:Toshihisa Tashiro% Done:

0%

Category:-
Target version:-

Description

[groonga-dev,00369] rroongaでのUInt32 VectorへのArrayオブジェクトの代入でSegmentation fault

rroongaを用いてのgroongaへのデータ挿入中、全体の9割ほど、数千万行をロードした辺りで
UInt32 VectorへのArrayオブジェクトの代入時にSegmentation faultが発生しました。
報告のため、以下にエラー発生箇所のスクリプトの抜粋とエラー時のメッセージを送付します。
よろしくお願い致します。

<テーブル定義>

 38 tweets      = Groonga::Hash.create( :name => "Tweets"    , :key_type => "UInt64")
 39 tweetIndex  = Groonga::PatriciaTrie.create( :name => "TweetIndex", :key_type => "UInt64")
 41 bigram      = Groonga::Hash.create( :name => "Bigram",    :key_type => "ShortText", :key_normalize => true, :default_tokenizer => "TokenBigram" )
 42 userIndex   = Groonga::Hash.create( :name => "UserIndex", :key_type => "UInt32" )
 53 tweets.define_column( "id", "UInt64")
 54 tweets.define_column( "posted_by", "UInt32")
 55 tweets.define_column( "point", "UInt32")
 56 tweets.define_column( "text", "ShortText")
 57 tweets.define_column( "favorites", "UInt32", :type => :vector )
 60 bigram.define_index_column( "text_index", tweets, :source => "Tweets.text", :with_position => true )
 61 userIndex.define_index_column( "user_index", tweets, :source => "Tweets.posted_by" )
 62 userIndex.define_index_column( "fav_index",  tweets, :source => "Tweets.favorites" )
 63 tweetIndex.define_index_column( "tweet_index",  tweets, :source => "Tweets._key" )




 27 def add_fav(tweet,user)
 28   tblTweet = Groonga["Tweets"]
 29   #p "add_fav "+tweet.to_s+" "+user.to_s
 30   if(tblTweet[tweet])then
 31     favs = tblTweet[tweet]["favorites"]
 32     if favs.index(user).nil?
 33       favs.push(user)
 34 
 35       point = favs.size
 36       if(favs.index(tblTweet[tweet]["posted_by"]))
 37         point -= 1
 38       end
 39 
 40       tblTweet[tweet]["favorites"] = favs
 41       tblTweet.set_column_value(tweet.to_i, "point", point )
 42     end
 43   end
 44 end



<エラー時ログ>

/home/matope/.rvm/gems/ruby-1.9.2-p0/gems/rroonga-1.0.1/lib/groonga/record.rb:68: [BUG] Segmentation fault
ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-linux]

-- control frame ----------
c:0013 p:---- s:0052 b:0052 l:000051 d:000051 CFUNC  :[]=
c:0012 p:---- s:0050 b:0050 l:000049 d:000049 CFUNC  :set_column_value
c:0011 p:0026 s:0043 b:0043 l:000042 d:000042 METHOD /home/matope/.rvm/gems/ruby-1.9.2-p0/gems/rroonga-1.0.1/lib/groonga/record.rb:68
c:0010 p:0140 s:0038 b:0038 l:000037 d:000037 METHOD insert_rroonga.rb:40
c:0009 p:0035 s:0030 b:0030 l:0008a8 d:000029 BLOCK  insert_rroonga.rb:125
c:0008 p:---- s:0027 b:0027 l:000026 d:000026 FINISH
c:0007 p:---- s:0025 b:0025 l:000024 d:000024 CFUNC  :each
c:0006 p:0365 s:0022 b:0022 l:0008a8 d:000021 BLOCK  insert_rroonga.rb:123
c:0005 p:---- s:0018 b:0018 l:000017 d:000017 FINISH
c:0004 p:---- s:0016 b:0016 l:000015 d:000015 CFUNC  :step
c:0003 p:0216 s:0011 b:0011 l:0008a8 d:001200 EVAL   insert_rroonga.rb:111
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:0008a8 d:0008a8 TOP
---------------------------
-- Ruby level backtrace information ----------------------------------------
insert_rroonga.rb:111:in `
' insert_rroonga.rb:111:in `step' insert_rroonga.rb:123:in `block in
' insert_rroonga.rb:123:in `each' insert_rroonga.rb:125:in `block (2 levels) in
' insert_rroonga.rb:40:in `add_fav' /home/matope/.rvm/gems/ruby-1.9.2-p0/gems/rroonga-1.0.1/lib/groonga/record.rb:68:in `[]=' /home/matope/.rvm/gems/ruby-1.9.2-p0/gems/rroonga-1.0.1/lib/groonga/record.rb:68:in `set_column_value' /home/matope/.rvm/gems/ruby-1.9.2-p0/gems/rroonga-1.0.1/lib/groonga/record.rb:68:in `[]=' -- C level backtrace information ------------------------------------------- /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9(rb_vm_bugreport+0x5f) [0x2ace7b9981df] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b88429b] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9(rb_bug+0xb3) [0x2ace7b884413] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b92cecc] /lib64/libpthread.so.0 [0x3bf6e0eb10] /usr/lib64/libgroonga.so.0 [0x3bf7674166] /usr/lib64/libgroonga.so.0(grn_ja_replace+0x360) [0x3bf767a510] /usr/lib64/libgroonga.so.0(grn_ja_putv+0x2b3) [0x3bf767b0c3] /usr/lib64/libgroonga.so.0(grn_obj_set_value+0x2267) [0x3bf76e5297] /home/matope/.rvm/gems/ruby-1.9.2-p0/gems/rroonga-1.0.1/lib/groonga.so(rb_grn_object_set_raw+0x122) [0x2aaaab0bbe22] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b9907f2] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9(rb_funcall+0x1da) [0x2ace7b991a8a] /home/matope/.rvm/gems/ruby-1.9.2-p0/gems/rroonga-1.0.1/lib/groonga.so [0x2aaaab0cb437] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b987211] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b989f2b] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b98f65d] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b992b28] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9(rb_yield+0x47) [0x2ace7b9932c7] /home/matope/.rvm/gems/ruby-1.9.2-p0/gems/mysql-2.8.1/lib/mysql_api.so [0x2aaaac620f48] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b987211] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b989f2b] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b98f65d] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b992b28] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9(rb_yield+0x47) [0x2ace7b9932c7] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b8c9595] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b987211] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b989f2b] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b98f65d] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9(rb_iseq_eval_main+0x1c7) [0x2ace7b98f927] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9 [0x2ace7b887482] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9(ruby_exec_node+0x19) [0x2ace7b8874a9] /home/matope/.rvm/rubies/ruby-1.9.2-p0/lib/libruby.so.1.9(ruby_run_node+0x2f) [0x2ace7b88973f] ruby(main+0x4f) [0x4008ef] /lib64/libc.so.6(__libc_start_main+0xf4) [0x3bf661d994] ruby [0x4007e9] [NOTE] You may have encountered a bug in the Ruby interpreter or extension libraries. Bug reports are welcome. For details: http://www.ruby-lang.org/bugreport.html

History

#1 Updated by Tasuku SUENAGA almost 8 years ago

C言語でリライトしてみたが、報告のものと関数実行パスが違う。 UVECTORではなく、VECTORとしないとgrn_ja_putv()に落ちないようだ。

VECTORでの値の設定の仕方がよくわからない。 grn_obj_open()で値を作るのだろうか。

#include 
#include 
#include 
#include 

#define SH_SIZE 32
#define CONST_STR_LEN(x) x, x ? sizeof(x) - 1 : 0

int
main(int argc, char *argv[])
{
  grn_ctx _ctx, *ctx;
  grn_obj _db, *db, *tweets, *user_index, *favorites, *fav_index;
  ctx = &_ctx;
  db = &_db;

  grn_init();
  grn_ctx_init(ctx, 0);
  GRN_DB_OPEN_OR_CREATE(ctx, "/tmp/tairyo_vector.db", NULL, db);
  GRN_TABLE_OPEN_OR_CREATE(ctx, "Tweets", sizeof("Tweets") - 1,
                           NULL,
                           GRN_OBJ_PERSISTENT | GRN_OBJ_TABLE_HASH_KEY,
                           grn_ctx_get(ctx, "UInt64", strlen("UInt64")),
                           NULL,
                           tweets);
  GRN_TABLE_OPEN_OR_CREATE(ctx, "UserIndex", sizeof("UserIndex") - 1,
                           NULL,
                           GRN_OBJ_PERSISTENT | GRN_OBJ_TABLE_HASH_KEY,
                           grn_ctx_get(ctx, "UInt32", strlen("UInt32")),
                           NULL,
                           user_index);
  GRN_COLUMN_OPEN_OR_CREATE(ctx, tweets,
                            "favorites", sizeof("favorites") - 1,
                            NULL,
                            GRN_OBJ_PERSISTENT | GRN_OBJ_COLUMN_VECTOR,
                            //grn_ctx_get(ctx, "UInt32", strlen("UInt32")),
                            user_index,
                            favorites);
  GRN_COLUMN_OPEN_OR_CREATE(ctx, user_index,
                            "fav_index", sizeof("fav_index") - 1,
                            NULL,
                            GRN_OBJ_PERSISTENT | GRN_OBJ_COLUMN_INDEX,
                            tweets,
                            fav_index);
  {
    grn_obj s;
    GRN_UINT32_INIT(&s, 0);
    GRN_UINT32_SET(ctx, &s, grn_obj_id(ctx, favorites));
    grn_obj_set_info(ctx, fav_index, GRN_INFO_SOURCE, &s);
  }
  {
    uint64_t seq;
    for (seq = 1; seq < 100000000; seq++) {
      int i;
      grn_id id;
      grn_obj vector;

      id = grn_table_add(ctx, tweets, &seq, sizeof(seq), NULL);
      if (id == GRN_ID_NIL) {
        printf("grn_table_add failed! %llu\n", seq);
        goto exit;
      }
      if (grn_table_size(ctx, tweets) != seq) {
        printf("table size is wrong! %llu\n", seq);
        goto exit;
      }
      GRN_UINT32_INIT(&vector, GRN_OBJ_VECTOR);
      for (i = 0; i < 100; i++) {
        GRN_UINT32_PUT(ctx, &vector, rand());
      }
      grn_obj_set_value(ctx, favorites, id, &vector, GRN_OBJ_SET);
      grn_obj_unlink(ctx, &vector);

      if (seq % 10000 == 0) {
        printf("%d added...\n", seq);
      }
    }
  }
exit:
  grn_obj_unlink(ctx, fav_index);
  grn_obj_unlink(ctx, favorites);
  grn_obj_unlink(ctx, user_index);
  grn_obj_unlink(ctx, tweets);
  grn_obj_close(ctx, db);
  grn_ctx_fin(ctx);
  grn_fin();
  grn_init();

  return 0;
}

#2 Updated by Tasuku SUENAGA almost 8 years ago

  • Tracker changed from Feature to Bug
  • Status changed from 新規 to 担当者作業中

#3 Updated by Tasuku SUENAGA almost 8 years ago

grn_obj_open()、grn_vector_add_element()を使うようにしたら動作した。 1件あたり5件のfavoritesを登録している。 これで再現を試みる。

#include 
#include 
#include 
#include 

#define SH_SIZE 32
#define CONST_STR_LEN(x) x, x ? sizeof(x) - 1 : 0

int
main(int argc, char *argv[])
{
  grn_ctx _ctx, *ctx;
  grn_obj _db, *db, *tweets, *user_index, *favorites, *fav_index;
  ctx = &_ctx;
  db = &_db;

  grn_init();
  grn_ctx_init(ctx, 0);
  GRN_DB_OPEN_OR_CREATE(ctx, "/tmp/tairyo_vector.db", NULL, db);
  GRN_TABLE_OPEN_OR_CREATE(ctx, "Tweets", sizeof("Tweets") - 1,
                           NULL,
                           GRN_OBJ_PERSISTENT | GRN_OBJ_TABLE_HASH_KEY,
                           grn_ctx_get(ctx, "UInt64", strlen("UInt64")),
                           NULL,
                           tweets);
  GRN_TABLE_OPEN_OR_CREATE(ctx, "UserIndex", sizeof("UserIndex") - 1,
                           NULL,
                           GRN_OBJ_PERSISTENT | GRN_OBJ_TABLE_HASH_KEY,
                           grn_ctx_get(ctx, "UInt32", strlen("UInt32")),
                           NULL,
                           user_index);
  GRN_COLUMN_OPEN_OR_CREATE(ctx, tweets,
                            "favorites", sizeof("favorites") - 1,
                            NULL,
                            GRN_OBJ_PERSISTENT | GRN_OBJ_COLUMN_VECTOR,
                            //grn_ctx_get(ctx, "UInt32", strlen("UInt32")),
                            user_index,
                            favorites);
  GRN_COLUMN_OPEN_OR_CREATE(ctx, user_index,
                            "fav_index", sizeof("fav_index") - 1,
                            NULL,
                            GRN_OBJ_PERSISTENT | GRN_OBJ_COLUMN_INDEX,
                            tweets,
                            fav_index);
  {
    grn_obj s;
    GRN_UINT32_INIT(&s, 0);
    GRN_UINT32_SET(ctx, &s, grn_obj_id(ctx, favorites));
    grn_obj_set_info(ctx, fav_index, GRN_INFO_SOURCE, &s);
  }
  {
    uint64_t seq;
    for (seq = 1; seq < 100000000; seq++) {
      int i;
      grn_id id;
      grn_obj *vector;

      id = grn_table_add(ctx, tweets, &seq, sizeof(seq), NULL);
      if (id == GRN_ID_NIL) {
        printf("grn_table_add failed! %llu\n", seq);
        goto exit;
      }
      if (grn_table_size(ctx, tweets) != seq) {
        printf("table size is wrong! %llu\n", seq);
        goto exit;
      }
      vector = grn_obj_open(ctx, GRN_VECTOR, 0, GRN_DB_VOID);
      for (i = 0; i < 5; i++) {
        int value = rand();
        grn_vector_add_element(ctx, vector, (const char *)&value, sizeof(value), 0, GRN_DB_UINT32);
      }
      grn_obj_set_value(ctx, favorites, id, vector, GRN_OBJ_SET);
      grn_obj_unlink(ctx, vector);

      if (seq % 10000 == 0) {
        printf("%d added...\n", seq);
      }
    }
  }
exit:
  grn_obj_unlink(ctx, fav_index);
  grn_obj_unlink(ctx, favorites);
  grn_obj_unlink(ctx, user_index);
  grn_obj_unlink(ctx, tweets);
  grn_obj_close(ctx, db);
  grn_ctx_fin(ctx);
  grn_fin();
  grn_init();

  return 0;
}

#4 Updated by Tasuku SUENAGA over 7 years ago

  • Status changed from 担当者作業中 to 新規

再現できていない。jaので修正されている可能性があるため、手放す。

#5 Updated by daijiro MORI about 6 years ago

  • Assignee changed from Tasuku SUENAGA to Toshihisa Tashiro

#6 Updated by Toshihisa Tashiro about 6 years ago

  • Status changed from 新規 to 担当者作業中

#3のプログラムの for (i = 0; i < 5; i++) { の部分を変えて(書き込むfavariteを増やして) 試したりしましたが、まだ再現しません。

#7 Updated by daijiro MORI about 6 years ago

  • Status changed from 担当者作業中 to 完了チェック待ち

再現しないのでいったん閉じます

#8 Updated by Toshihisa Tashiro about 6 years ago

  • Status changed from 完了チェック待ち to 完了

Also available in: Atom PDF