I am guessing the width and height of the video file are quite large. If you plop it down to a size suitable for gif, say maybe 400px width, you'll see a massive drop in filesize.