Snapshot Image Downloads in Features and Ideas

2

Comments

  • edited May 2015
    Baaah, we pay for an unmanaged service, it's not hard to create your own snapshots!

    Ok, this would improve the functionality, for sure, and a web based button may be handy for some (though I'd never rely on a browser to download something so big!), but it's patently not "literally" impossible!
  • Please do let me know how I can create my own snapshot of a Windows Server? :)
  • edited May 2015
    @RichardA damn, you called my bluff! I suspect "replace windows with FreeBSD" is not a satisfactory answer? :-)

    Seriously, though, how about this:

    If you don't want downtime, take a snapshot of your instance, and restore it to a new instance. If you don't mind downtime, stick with your current instance.

    Next, power off the instance, and in the control panel, select and load a custom ISO. (Prior to this, upload a "live cd" to your ISO area, I use FreeBSD-10.1-RELEASE-bootonly.iso but I'm sure the linux distros would work too)

    Boot off that iso, then select 'shell' / 'live cd' rather than install.

    Configure the network, and download the raw device. The instances disk will have remained untouched and unmounted during this time:

    dd if=/dev/vtbd0 bs=1m | gzip -c | ssh -e none myuser@myhost 'cat > mysnapshot.iso.gz'

    Bingo, snapshot compressed and downloaded. Of course, you may remove the 'gzip -c |' bit, or maybe replace it with a ZIP/rar equivalent, etc. but you get the jist!

    It's quicker to do than it is to write!, I've used a similar variation a few times...


  • +1

    If you can provide a way to download a image will be great for Vultr, as a i new customer will be a kick ass feature is if can download my server image and use it somewhere else.
    Will make feel you're so sure of your products that you don't have to tie the users servers to keep them
  • +1 for this but it may be a billing issue for them so I would suggest adding the download option but at maybe $0.05 or $0.10 per GB.
  • +1 and for large storage as well
  • +1
    Any updates on this feature?
  • +1 Also: The ability to UPLOAD *and* DOWNLOAD snapshots would be very helpful. Currently, Vultr is expiring and deleting my snapshots after a certain time. I need to maintain different snapshots for different client VPS systems, at various stages of functionality (from BASELINE, to CONFIGURED, to PRODUCTION) and cannot do so if they're deleted after an expiration period.
  • +1 This would be a great feature.

    @jamie - would care to do a tutorial on youtube pls?
  • I need to move my server from Dallas (because it is full now) to Chicago. How can i do that?
  • @jamie I'm running CentOS 7 x64 and I tried to do what you said in order to get a snapshot "download".

    But.... well... I failed.
    Would you be able to make a tutorial in video and upload it on youtube?
  • I'm for image upload. It would be great to have systems like Chrome OS and Hackintosh on a VPS.
  • Sorry guys, I've not been around much this year!

    OK, I'll do a video... Bear with me - it won't be polished or editted, and as it's over a year ago I last did this, I'm a bit rusty!

    Sorry for the mouse pointer jitter - I'm doing this using an 'air' remote control (like a wii controller with a keyboard) on my android tv!
  • edited August 2016
    @Dylan @diego I forgot to include the URL!



    Let me know how it goes!

    @dac6acc4f - You can do that using the technique I posted, but in reverse. However I'm sure you could retrieve what you want from Vultr's very own 'install from custom ISO' option.

    @vladi Make a snapshot of your instance, then create a new instance in Chicago, using 'install from snapshot' option.
  • P.S. Read the notes in the youtube video description before you start!
  • @jamie OH! Thanks for the video and sorry for the late response.
    I'll try it and I'll give you a feedback on the video :)

    Thanks again!
  • @jamie To restore I just upload as an ISO to the vultr panel, right?
  • edited September 2016
    @diegp Yeah. You end up with the same snapshot type as created by the Vultr setup.

    Firstly, for the pedants, I refer to 'disk' and 'cd' etc. because that's how they behave. I know they are all virtual devices under KVM, but it's clearer to refer to them as the device they are emulating.

    Basically, when you do a Vultr snapshot, Vultr freezes all operation on your vhost, copies the raw disk file, then unfreezes the vhost. Well, that's effectively what is done - However, literally doing it that way would mean your vhost is frozen for some time!

    I'm guessing that what will actually be done will be presumambly as follows:

    1) The disk access in the vhost is frozen:

    Then either of these 2 methods:

    2a) Either a COW (copy on write) copy will be made - the disk file is 'copied' by creating a file that links to all the data blocks in the original file. This can be generally done in a second or so.

    That file is basically a snapshot, because if anything changes on the vhost, the old data is copied into the snapshot file - i.e. the file is modified invisibly as stuff on the vhost changes, thus the contents of the file will never change from when the snapshot was taken (see FreeBSD snapshots)

    2b) Journalling is enabled. This is basically similar to the above, but the other way around - the original disk file is not written to. Any subsequent files changes are recorded in a 'journal' which simply shows what's changed betwwen the current snapshot and your live system

    Again, this is invisible to the vhost.

    3) Disk access is unfrozen. The total frozen time is in seconds, if that.

    Which ever method is used, you end up with the same thing: a static frozen coherent copy of your vhost. This file is then simply copied to archive, and the COW file deleted, or the journalling information being applied and the journaling reset.

    Again, this is invisible to the vhost.

    As we don't have access to these functions via the main vultr host-box, we get our snapshot of the vhost by basically shutting it down, then effectively booting your 'machine' off a live CD, which means the vhosts 'disk' is not active.

    Whilst booted off this CD, we simply copy the frozen vhost disk fille, in this case via the network, as we have nowhere else to store it. This is why it will work with any OS. The live-cd doesn't mount the partitions. It doesn't care if the partitions are ext4/ufs/zfs/FAT/NTFS etc. It just copies the whole 'disk' as one big raw file.

    The point is, you end up with the same result: a coherent snapshot of your system (it's possible to do this live via the vhost without shutting it down, but this is seriously not recommended!)

    I forgot to say, if you have more than one 'disk' (*disk* not *partition*) inside your vhost, then obviously you'll need to repeat this process for those too.

    TL; DR:

    Finally, don't trust some random git like me you only know from online. Also don't trust that you'll always get it right first time.... I'd seriously recommend restoring your instance to a temporary test-vhost to ensure everything has worked smoothly.
  • Hi, thanks for suggesting the FBSD option for snapshot transfer, but I'd like to know if there is a friendlier way of doing the task, like this:

    https://www.digitalocean.com/company/blog/easily-transfer-snapshots-between-accounts/
  • @jamie do you know if I make this backup, I can migrate to other company?
    I'm thinking about moving to OVH (7$, more resources) but import all my stuff.

    Do you know if it will be ready-to-go as soon as I import the "disk-backup" ?
  • @jamie Well... my dev folder is empty. I couldn't use this command.

    Take a look:
    http://image.prntscr.com/image/eb9b58d65c5444cd8f4a1a0b7ffe641f.png

    First time it said bs=1m was wrong, so I replaced with bs=1M
    Then it said: /dev/vtbd0 not found.

    So I went to file system to look the /dev/ folder and... it was empty O_o.
    I tried to run file system with elevated privilegies and still empty.

    Can you help me?
  • btw: My home folder is there with all my files, so I'm sure this is the correct device to look at.
  • @diego Ummm.. OK, you've raised a number of points here, and I'm a bit confused to what you've done, so pldase bare with me.

    Firstly, I didn't make this post as a way to make it easier to leave Vultr, and you have a bit of a cheek asking on a Vultr forum!!

    Having said that... There may be people who will use Vultr more if they are able to hold snapshots personally.... hmmmmm.

    I have no idea how OVH systems work, but maybe if I try to make it clearer to what a snapshot is.

    Imagine you have a home PC, and the hard disk is making a funny sound - but still working. Worried, you want to replace the disk.

    If you get an identical sized replacement disk, this snapshotting process is basically making a clone - so you end up with a new drive that's identical to the first. That means, the same OS configurations, the same file order, any windows partition being just as fragmented as before, that big dodgy file you deleted a while back that hasn't yet been overwritten with new data still "existing in the shadows"......

    You can then replace that disk and everything acts the same.

    Or, you could take the new disk to a mates house, with a *similar spec. PC*, swap the drives over, and there would then be a direct copy of your disk.

    Now, the way you made that copy is irrelevent. If you use a hardware disk cloaner, or use FreeBSD, Linux, windows, VAX/VMS, multics, the first Babage computer, or a bunch of guys with tweasers, a hammer, and a magnifying glass, the END RESULT is the same.

    That's why I suggested FreeBSD - it's the system I have the most knowledge of, and it works.

    You said bs=1m didn't work. That makes me assume you are using Linux to do the transfer. That's fine (as I said, what tool you use is irrelevent)

    1m is shorthand for 1 megabyte. 1k is 1 kilobyte. Linux doesn't have the 'm' suffix, so on Linux you'd instead use bs=1024k

    Having said that, the value really only affects transfer efficiency, and probably even less so when your "disks" are virtual. On top of that, 1MB is probably way larger than practically necessary!)

    [ EDIT: I just looked at a linux system, and you're right - GNU dd uses 'M' instead of 'm'. However, I still feel vindicated - I said I was describing a process, and my examples were based on FreeBSD, and if you choose to use a different system, then you need to be aware of subtle differences ]

    But somewhat related - I'm assuming that screenshot is from a Linux desktop - /dev/vbd0 is the *Freedbsd* naming scheme for a *virtio virtual disk*

    The name will be different under linux, and even under freebsd if you restore the snapshot to a real hard disk, or a mountable file image etc.

    And again, the /dev/vbd0 is just for the snapshotting process - you don't access it via the filemanager, if you are testing the snapshot on your local system, you need to mount the image (from memory, I thin Linix calls it loopback, and you need to use losetup - but I could be wrong there)

    So, it's basically mounted, as you would (say) a USB stick, and the files on the filesystem accessed in a similar way.

    It's not clear to me from your desktop screenshot what you are doing : have you at this stage downloaded a big snapshot file you want to view? Why are you trying to access vdb0?

    Oh, and if /dev is empty, you have far more problems than I can solve! - incidentally, I'm sure some of the Linux kernel developers would be interested in how the hell your system runs! :-) [ my guess is that your file-manager doesn't display block/character specials ]
Sign In or Register to comment.