Uploaded image for project: 'Project FiFo'
  1. FIFO-929

Deleting VM's with backups and s3 errors

    Details

    • Type: Improvement
    • Status: Resolved (View workflow)
    • Priority: High
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 0.7.2
    • Component/s: sniffle
    • Labels:
      None

      Description

      This is a bit of a tricky one.

      When the s3 delete fails the delete command fails, neither option seems good

      1) if we delete anyway we risk that we have abandoned files inside of s3
      2) if we block the delete we can't delete VM's after a s3 crash.

      This should be solved in a sensible manner, not sure which yet

      Exception in FiFo Test Lab

      unknown in sniffle@192.168.1.41
      Process <0.32323.190> with 0 neighbors crashed with reason: {{aws_error,
      {socket_error,
      {failed_connect,
      [{to_address,{"fifo-backups.192.168.1.42.xip.io",8443}},

      {inet,[inet],etimedout}

      ]}}},
      [{erlcloud_s3,s3_request,8,
      [

      {file,"/root/sniffle/_build/default/lib/erlcloud/src/erlcloud_s3.erl"},
      {line,855}]},
      {erlcloud_s3,delete_object,3,
      [{file,"/root/sniffle/_build/default/lib/erlcloud/src/erlcloud_s3.erl"}

      ,

      {line,203}

      ]},
      {sniffle_vm,'do_delete_backup/3-lc$^0/1-0',1,
      [

      {file,"/root/sniffle/_build/default/lib/sniffle/src/sniffle_vm.erl"},
      {line,1259}]},
      {sniffle_vm,do_delete_backup,3,
      [{file,"/root/sniffle/_build/default/lib/sniffle/src/sniffle_vm.erl"}

      ,

      {line,1259}

      ]},
      {sniffle_vm,'finish_delete/1-lc$^0/1-0',3,
      [

      {file,"/root/sniffle/_build/default/lib/sniffle/src/sniffle_vm.erl"},
      {line,840}]},
      {sniffle_vm,finish_delete,1,
      [{file,"/root/sniffle/_build/default/lib/sniffle/src/sniffle_vm.erl"}

      ,

      {line,840}

      ]},
      {sniffle_tcp_handler,message,2,
      [

      {file, "/root/sniffle/_build/default/lib/sniffle_api/src/sniffle_tcp_handler.erl"}

      ,

      {line,449}

      ]},
      {mdns_server_protocol,handle_data,3,
      [

      {file, "/root/sniffle/_build/default/lib/mdns_server_lib/src/mdns_server_protocol.erl"}

      ,

      {line,109}

      ]}]}

      View on Bugsnag

      Stacktrace

      /root/sniffle/_build/default/lib/erlcloud/src/erlcloud_s3.erl:855 - s3_request
      /root/sniffle/_build/default/lib/erlcloud/src/erlcloud_s3.erl:203 - delete_object
      /root/sniffle/_build/default/lib/sniffle/src/sniffle_vm.erl:1259 - '-do_delete_backup/3-lc$^0/1-0-'
      

      View full stacktrace

        Attachments

          Activity

            People

            • Assignee:
              heinz Heinz N. Gies
              Reporter:
              bugsnag Bug Snag
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: