Author Topic: HippoEDIT 2.00  (Read 1101 times)

Offline alex

  • Developer
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2224
  • Karma: +37/-3
    • HippoEDIT
HippoEDIT 2.00
« on: May 09, 2015, 09:49:50 PM »
New major update of the HippoEDIT will get version 2.00. Below you can find links to first alpha builds and can see what is planned to be developed:

HippoEDIT 2.00 for Win 32 bit (Unicode)
HippoEDIT 2.00 for Win 64 bit (Unicode)
HippoEDIT 2.00 for Win 32 bit (Non-Unicode) - I think, it shall be deprecated. Usage percentage is very small, but the effort to support is significant.

*All installers contain PDB files for better crash reporting, so sizes are rather big (approx 37 mb), but this does not influence performance of the application itself.

Important: from version 2.00 HippoEDIT will switch to new licensing model:
  • Price of the Standard version will be decreased.
  • Published mode with 1 year of technical support and free updates for new minor version will become official. The standard license will be valid only for major version active in the time of purchase, can be updated free to new minor version and will allow an upgrade to next major version during the first year after purchase
  • There will be LifeTime license, which will be valid for all next minor/major version without any limit to the time of purchase (the mode which was unofficially active till now). LifeTime version will cost more than Standard version. All licenses purchased before publication of HippoEDIT 2.00 will be treated as LifeTime licenses.
  • There will be Standard versions, bound to some specific interface language, will cost even less than Standard version.
  • There will be NO XP support, and probably also no support for non-Unicode version

If it crashes on startup or says that some DLLs not found, you need to install Visual Studio 2017 redistributables. They can be found on Microsoft site - take version passing to your HippoEDIT architecture (x86 or x64).

Following changes are planned for HippoEDIT 2.0:
  • Minor updates of UI making HippoEDIT even more flat and concentrated on content.
  • Change of licensing model.
  • Performance optimizations for start up, search and closing
  • Animated selections
  • Full support of high DPI Windows settings
  • HippoEDIT as source code file preview in Windows Explorer and MS Outlook attachments
  • Opening of images, office documents, PDFs in HippoEDIT in read-only mode
  • Hex Editor plugin.
  • Total Commander Lister Plugin => TCViewer.wlx(64)
  • Scripting for Code Completion
  • Code Completion: new ranking logic, filtering by substring, acronym or shorthand, completion based on content of other open documents
  • UI Localization support, with support of user contribution for localization
« Last Edit: June 21, 2018, 02:13:31 PM by alex »

Offline S.H.Bouwhuis

  • Registered User
  • Newbie
  • *
  • Posts: 27
  • Karma: +0/-0
Re: HippoEDIT 2.00
« Reply #1 on: March 06, 2017, 04:34:46 PM »
I'm interested in the hex editor plugin.

I was wondering,  how are newer builds released?
This thread is from 9-May-2015, I presume there are newer builds?

Offline alex

  • Developer
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2224
  • Karma: +37/-3
    • HippoEDIT
Re: HippoEDIT 2.00
« Reply #2 on: March 07, 2017, 12:17:44 AM »
Hi Sander,

the last build update was from 4.12.2016 (2.00.36 build), I just have not updated the post, because there were no significant new features.
But I have updated the build also today (to build 2.00.37), just because of your request ;)

In current the state the Hex Edit plugin is still not yet finished, but already allows modification operations and a lot of editing functionality as:
- editing in different number base modus
- Copy/Pasting in text and binary format
- Drag & Drop editing
- Saving of the selection / creating new file from clipboard data or selection
- Creating of new Hex Documents
- Persisting of Untitled Hex Documents without explicit saving on editor close
- Undo/Redo (history is still not visualized)
- Goto functionality
- Opening in text/hex mode
- yet buggy indicator of changed bytes
- etc.

What is not yet in, but shall come to the first release of Hex Plugin:
- Find/Replace in Hex Document (probably only in current document)
- Visualization of Undo history (list of actions with mass undo/redo)
- Merging of Undo/Redo neighbor actions
- Context menu
- Bookmarks

Best regards,
Alex.

Offline S.H.Bouwhuis

  • Registered User
  • Newbie
  • *
  • Posts: 27
  • Karma: +0/-0
Re: HippoEDIT 2.00
« Reply #3 on: March 07, 2017, 09:21:39 AM »
It sounds like you are approaching completion to a degree that it is actually useful to use now.

You state that pasting and undo work in hex editor mode:

1. Do you mean binary pasting as inserting (great!), or pasting as overwriting (meh)?
2. Undo, so that means a mode where you don't immediately commit to disk (i.e., you buffer the binary data)?

With Notepad++ having deprecated the hex editor and Ultra Edit being slow to start, HippoEdit is starting to look like an interesting replacement.

Offline alex

  • Developer
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2224
  • Karma: +37/-3
    • HippoEDIT
Re: HippoEDIT 2.00
« Reply #4 on: March 07, 2017, 10:48:47 AM »
Quote
1. Do you mean binary pasting as inserting (great!), or pasting as overwriting (meh)?
The editor supports both modes, overwriting and inserting of data. Overwriting is a default mode (switched by Ins).
But, I think now pasting only works as inserting, in both modes. Of course, if you paste with selected data, it will be overwritten. But it works as: first delete, then insert. Probably, it would be good to add real overwrite, with auto increasing of the file, if data to be pasted - larger than available space.

Quote
2. Undo, so that means a mode where you don't immediately commit to disk (i.e., you buffer the binary data)?
HippoEDIT never immediately commits on disk - I found it to be too dangerous for working mode. You always need to press Save to get your changes on disk. Editor remembers your modifications of the original data (altered parts of the file duplicated to the operating memory), and flushes it on save. Undo/Redo is just a history of your edits you can reply on modification buffer. In the current state, Undo history is not persisted between sessions (as it works for the text editor), so you can not use old Undo/Redo after a restart of the editor.

BR, Alex.


Offline S.H.Bouwhuis

  • Registered User
  • Newbie
  • *
  • Posts: 27
  • Karma: +0/-0
Re: HippoEDIT 2.00
« Reply #5 on: March 07, 2017, 11:55:47 AM »
The editor supports both modes, overwriting and inserting of data. Overwriting is a default mode (switched by Ins).
But, I think now pasting only works as inserting, in both modes. Of course, if you paste with selected data, it will be overwritten. But it works as: first delete, then insert. Probably, it would be good to add real overwrite, with auto increasing of the file, if data to be pasted - larger than available space.
So, I have 0x000102030405060708090a0b0c0d0e0f and I insert 0xaabbccddeeff at the 5th byte would result in:
0x00010203aabbccddeeff0405060708090a0b0c0d0e0f
Correct?
HippoEDIT never immediately commits on disk - I found it to be too dangerous for working mode. You always need to press Save to get your changes on disk. Editor remembers your modifications of the original data (altered parts of the file duplicated to the operating memory), and flushes it on save. Undo/Redo is just a history of your edits you can reply on modification buffer. In the current state, Undo history is not persisted between sessions (as it works for the text editor), so you can not use old Undo/Redo after a restart of the editor.
Although that is great in principle, how would you handle it when a file is a couple of terabytes (i.e., MUCH larger than system memory)?

Combining both points, how does HippoEdit insert some bytes in a very large file?

Offline alex

  • Developer
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2224
  • Karma: +37/-3
    • HippoEDIT
Re: HippoEDIT 2.00
« Reply #6 on: March 07, 2017, 01:41:36 PM »
Quote
So, I have 0x00102030405060708090a0b0c0d0e0f and I insert 0xaabbccddeeff at the 5th byte would result in:
0x0010203aabbccddeeff0405060708090a0b0c0d0e0f
Correct?
Yes, it is like you described now. And it is correct for "insert" mode. For overwrite mode it shall be like this:
0x00102030aabbccddeeffa0b0c0d0e0f

But honestly, as I have checked, even for the text editor (in HippoEDIT), it inserts always. So, it is consistent :) But I still think, that proper way will be to overwrite of the content by pasted block... Need to check how it is done by others.

Quote
Although that is great in principle, how would you handle it when a file is a couple of terabytes (i.e., MUCH larger than system memory)?
I do no load a whole file into a memory. Only changed parts (some snapshots of the original file, that were modified). Not modified parts are read directly from the file.
Buf of course if one would like to modify every byte of terabyte file, it may be problematic. But still will work, at least on x64 architecture. I would not allocate one big block of terabyte size, but a lot of small ones. And operating system will use a swap file to support that.

Quote
Combining both points, how does HippoEdit insert some bytes in a very large file?
Inserting works in memory, HippoEDIT will load original memory block around the place you modify, insert in it the bytes, resize it, resize in memory all offsets to original data for next blocks. When saving, HE checks if the file was resized (if not, original data is directly overwritten), then double resize the file, write new content to end of the file, then move new content on top of the existing data and resize the file to the new size.

BR, Alex.

Offline S.H.Bouwhuis

  • Registered User
  • Newbie
  • *
  • Posts: 27
  • Karma: +0/-0
Re: HippoEDIT 2.00
« Reply #7 on: March 08, 2017, 03:14:18 PM »
Do you literally double the file size on save?
That would be a problem for me when working with files in the high GB / low TB range.

Why not simply increase the file size by say 1MB and then move the existing bytes 1MB at a time up until you reach the newly inserted bytes?

Offline alex

  • Developer
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2224
  • Karma: +37/-3
    • HippoEDIT
Re: HippoEDIT 2.00
« Reply #8 on: March 08, 2017, 04:25:14 PM »
Yes, I know that "doubling" the size is not an ideal solution, but it is easiest and ensures an acceptable performance on save. While otherwise, you may do more of move operations: imagine you insert 1-byte in front than once more 1 byte after 1 MB interval etc. You will need to do shifts of rest of file twice then.
Another example: you have inserted 1 MB in front of the file and deleted 2MB at the end of the file and then inserted 1 MB at the end of the file. So file size is same and now you need to invent an intelligent logic how to do all modification without overwriting "untouched" memory blocks and minimizing memory moves.

So, yes, currently is not ideal, but may work, if you have enough space on your drive. In next versions, I will evaluate if more efficient logic can be applied.

Offline S.H.Bouwhuis

  • Registered User
  • Newbie
  • *
  • Posts: 27
  • Karma: +0/-0
Re: HippoEDIT 2.00
« Reply #9 on: March 08, 2017, 11:54:57 PM »
I'm impressed you keep track of the changes and then apply them one by one when saving. That is not easy to handle.

I have a suggestion which would make things simpler and (most importantly) safer:
Simply create a new secondary file.

Advantages:
1. If something goes wrong (power loss, shut down, crash, disk full, original file moved or deleted, ...), the user still has the original file.
2. You can create the file in one go at the correct size.

Especially the first point is in my opinion ESSENTIAL.

PS
I didn't really check yet, but is HippoEdit open source?

Offline alex

  • Developer
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2224
  • Karma: +37/-3
    • HippoEDIT
Re: HippoEDIT 2.00
« Reply #10 on: March 09, 2017, 05:22:14 PM »
Yes, the solution with secondary/temp file on saving with following replace the original file by temp file is more secure.
But... you lose all security attributes of original file :/ And if in a case of Windows you can somehow copy it (also not trivial), for not Windows platform, when remote files are edited, it mostly impossible. So, the only way to keep original security attributes of the file is to work directly on it. Resizing and writing data at the end of file is not very critical (if it crashes, the original data is still there), moving of data over is critical, but it is an always a risk.

Having a backup of the original file is an option.

HippoEDIT uses the same technic for saving of text files. Plus it saves every 500 ms you changes (aside to original file) and restores them in a case of a crash.

HippoEDIT is not open source. Only some plugins are open source.

BR, Alex.

Offline S.H.Bouwhuis

  • Registered User
  • Newbie
  • *
  • Posts: 27
  • Karma: +0/-0
Re: HippoEDIT 2.00
« Reply #11 on: March 09, 2017, 10:09:04 PM »
Hmmm, can't you use the CreateFile function to use the security attributes of another file?

https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx

Offline alex

  • Developer
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2224
  • Karma: +37/-3
    • HippoEDIT
Re: HippoEDIT 2.00
« Reply #12 on: March 10, 2017, 12:29:06 AM »
You can, but with restrictions:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa446654(v=vs.85).aspx
And this does not work with remote files from not NTFS volumes. So, when you access remote files from Unix based systems for example.
Plus there maybe some other data, that one need to copy also and that easy to overlook, something like alternative NTFS streams, for example.

So, my experience here was, that easiest and most consistent way is to work on the original file.

Offline S.H.Bouwhuis

  • Registered User
  • Newbie
  • *
  • Posts: 27
  • Karma: +0/-0
Re: HippoEDIT 2.00
« Reply #13 on: March 10, 2017, 12:39:05 PM »
Ok, got it.

Thanks for the indepth information.

Offline alex

  • Developer
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2224
  • Karma: +37/-3
    • HippoEDIT
Re: HippoEDIT 2.00
« Reply #14 on: October 19, 2017, 12:27:16 AM »
A new build of HippoEDIT 2.00 published.

HexEdit Plugin is finished, with new functions, as:
- Find/Replace in Hex Document (only in current document) with Find All log
- Visualization of Undo history (list of actions with mass undo/redo)
- Merging of Undo/Redo neighbor actions
- Context menu
- Bookmarks (integrated into Bookmark Manager plugin)
- Visualization of Modified bytes
- Switch between Hex/Text mode

Once more performance optimization for:
- startup
- search
- word wrap
- file previewer mode

A new feature for autocompletion:
- Auto Acronyms for long, camel case keyword as Code Templates

Links for download in topic header.