Data migration route
Amazon S3 data to SQLite
Amazon S3 is cloud file storage, and SQLite is a store of data kept as files the firm holds. What lands in SQLite is the register: every file’s name, place, dates and details, with the files themselves stored beside it.
Amazon S3Cloud- Documents: Exported with mapping
- Emails: Exported with mapping
- Custom fields: Exported with mapping
SQLiteFiles- 3 mapped
What moves
What moves from Amazon S3 to SQLite
Each row is a kind of record Amazon S3 holds, whether SQLite has a place for it, and what is planned. On this route three kinds need mapping.
| Record | What is planned | ||
|---|---|---|---|
| DocumentsFiles stored in or linked from the system | HeldObjects in buckets; a folder is only the part of the key before a slash. | Has a place in partListed in a table; the files themselves stay in folders beside the database. | Exported with mapping |
| EmailsEmails saved to matters or to the store | Held in partOnly where messages were saved to the bucket as files such as .msg or .eml. | Has a place in partAs message files in folders, indexed in a table. | Exported with mapping |
| Custom fieldsFields and record types the firm added itself | Held in partObjects can carry user-defined metadata and tags as name and value pairs. | Has a placeColumns added to the table they belonged to. | Exported with mapping |
What the plans mean
- Exported with mapping
- SQLite holds this differently, or in part. Each field or value is matched to its place before loading, and you see the mapping first.
The table states what the two products hold and what NewLaw plans for. It is not a promise that every record moves: that is settled against your own data in the analysis.
This kind of move
The files, and a register of what each one is.
A database is a poor place to keep documents and a good place to keep facts about them. On this route the files from Amazon S3 are exported to storage the firm controls, and SQLite receives the register: one row per file, with its path, size, dates, author and whatever profile or folder details existed. That register is what makes a large export searchable. It also proves completeness, since the row count and total size can be agreed to the source.
How useful the register is depends on how much Amazon S3 knows about each file. A document management system has a profile to carry. A plain folder has only a path and dates, and the client or matter has to be read out of folder names. Where versions or an audit history exist, each becomes a table of its own. The export is checked by picking documents at random in the source, finding them through SQLite, and opening the file to confirm it is the same one.
Amazon S3 is hosted by its maker and SQLite sits on equipment the firm controls. The data leaves a service that was backed up for the firm and arrives somewhere the firm must back up itself, so backup and access are arranged before the export is copied in. What Amazon S3 will release depends on its export tools, and that is confirmed in the analysis.
Where the data is now
Leaving Amazon S3
Moving a firm’s files out of S3 starts with finding out who put them there and how they are named. A bucket filled by a person usually mirrors a client and matter folder tree. One filled by an application may hold nothing but numbered objects, with the real file names kept in that application’s database. Those are different jobs, and the analysis says which one this is.
Folders are part of the name
S3 has no folders. A key such as Clients/Nguyen/2019-0412/Letter.pdf is one long name, and the console draws folders from the slashes. Keys can run to 1,024 bytes, far longer than Windows or SharePoint accept, and may contain characters they reject. Those keys are listed in the analysis and shortened by an agreed rule.
Modified dates are upload dates
The date S3 records against an object is the date it was written to the bucket. A letter drafted in 2014 and uploaded in 2021 shows as 2021. Unless the original date was kept in object metadata or in an index, it cannot be recovered from S3, and document dates in the new system will reflect that.
Archive classes need a restore
Objects in the S3 Glacier Flexible Retrieval and Deep Archive storage classes cannot be read straight away. Each has to be restored to a temporary copy first, which takes time and is charged by AWS. Closed-matter archives are often stored this way, so the restore is requested before the trial load, not during it.
Versions exist only if enabled
Versioning is off unless someone switched it on for the bucket. Where it is on, every overwrite keeps the earlier object in full, and a deleted file sits behind a delete marker and is not removed. The firm decides whether earlier versions and deleted objects move, because a plain copy of the bucket brings only the current ones.
Application buckets need the database
Where a practice or document system used the bucket as its store, the keys are often identifiers and not file names. The matter, the document title and the author live in that system’s database. Copying the bucket alone produces thousands of unnamed files, so the objects are matched to their records before anything is loaded.
Where it is going
Arriving in SQLite
A SQLite archive suits a firm that wants the old system’s records kept, seldom consulted, and free of any server or subscription. Everything structured goes into one file. That simplicity moves the responsibility elsewhere: the file has no security of its own and no administrator watching it, so where it is stored and who can reach it are the real decisions.
One file, kept with the archive
Matters, contacts, time, invoices and trust transactions become tables in a single file. It is stored with the firm’s closed-matter records and can be copied like any document. A checksum recorded when the file is finalised lets anyone confirm, years later, that the copy they hold has not been altered, and the firm decides whether to keep one.
No logins inside the file
SQLite has no users or passwords. Anyone who can open the file can read every client’s ledger. Protection comes from where it is kept, such as a restricted folder or an encrypted volume, and that choice is the firm’s. It can suit an archive consulted by two people, and it is said plainly so nobody assumes otherwise.
Built for reading, not sharing
Only one program can write to the file at a time, and SQLite’s authors point people to a client/server database where many users share data over a network. A finished archive never meets that limit. It is the reason SQLite is used here for records that are closed and not for anything staff still update.
Documents stay outside it
Documents remain ordinary files in matter folders, and a table in the database lists each one with its matter, title, dates and path. Keeping them as files means they open with a double-click and can be handed to a client. The table is how a file is found by matter number instead of by guessing its name.
Someone has to open it
SQLite publishes its own command-line program for opening a database file, and other tools read the format too. None of that helps if nobody remembers how. Saved queries for the usual look-ups and a short note on which tool to use can sit in the same folder as the file, and the firm decides what it needs.
How NewLaw runs it
Four steps, in this order.
This route produces an archive, not a working system, so the steps are about completeness and proof.
Analysis
We read what Amazon S3 holds, table by table, and agree with you what the copy in SQLite must be able to answer later. Everything Amazon S3 holds is in scope unless you take it out.
Clean-up, if you want it
An archive should show the records as they were, so nothing is merged or corrected unless you ask. Where the copy is also meant for reporting, duplicate and inconsistent entries are tidied in a working copy and the original values are kept beside them.
Trial export
The data is written to SQLite as a trial and checked against Amazon S3: record counts, totals where Amazon S3 holds money, the links between records, and a sample of documents opened from the export. Exceptions are listed for you to decide.
Final export
After the last day of work in Amazon S3, the export is run a final time and checked again. There is nothing to go live. Amazon S3 is kept read only until you are satisfied the copy answers what you need, and it is handed over with a written description of its contents.
Fixed price, agreed after the analysis. How NewLaw moves legal data
Start
Start this migration
Tell us what is in Amazon S3 and roughly how much of it. The first step is the analysis, and the price is fixed once it is done.
Step 1 of 5
What happens after you send it
- A free first call
We go through your answers with you and ask about anything they leave open.
- An analysis of what Amazon S3 holds
What is in it, how its records link, and where each piece would sit in SQLite. Anything with no place there comes across as a note or stays behind, and you see which before the move.
- A fixed price in writing
Set after the analysis, so it reflects what is in your data. You have it before any work starts.
Beyond the data
Structure, permissions and handover
SQLite is a place to keep data, not a system staff work in each day. What is agreed is how the data is laid out, who can reach it, and what is handed over with it.
There is nothing to install on a server and nothing to license. The work is in making the file usable and safe to keep: naming it, documenting its tables, storing it where the firm’s records policy says archives go, and checking that a member of staff can open it and find a matter. Whether a second copy is held somewhere separate is the firm’s call.
What is agreed and set up
- File name, storage location and whether a second copy exists
- Whether a checksum is recorded for later verification
- Saved queries for matter, ledger and document look-ups
- Which tool staff will use to open the file
- Who may open the folder the file sits in
- Retention period and destruction date noted beside the file
Questions
Amazon S3 to SQLite: what firms ask
Our IT provider set the bucket up. What do you need from them?
Access to read the bucket, arranged in whatever way the firm and its provider decide, and the region it sits in. They are also asked whether versioning is on, whether any objects are in an archive storage class, and whether a person or an application wrote the files. Moving the files out means reading them, so write access is not part of the request.
Will our client and matter folders look the same in the new system?
In outline, if the keys follow a client and matter pattern. Each part of the key between slashes becomes a folder, or is read as the client, the matter number and the document name where the destination files by matter. Keys that fit no pattern are listed for the firm to place by hand or leave in an unsorted folder.
Can the bucket stay as our archive after the move?
It can. A firm may move its active matters into the new system and leave closed files where they are. What matters then is that someone at the firm, not only the IT provider, holds the AWS account details and knows how to find one closed file by client name. That is worth writing down before the project ends.
Does the copy in SQLite hold documents from Amazon S3?
In part. Listed in a table; the files themselves stay in folders beside the database. What SQLite can hold is written out once its mapping is agreed, and the rest is kept beside the data as files that open without Amazon S3.
Does everything in Amazon S3 have a place in SQLite?
On paper, yes: each kind of record Amazon S3 holds has a place in SQLite. Whether every field fits is a separate question, and the analysis answers it field by field. Anything with no place comes across as a note or stays behind, and you see which before the move.
What has to be decided before Amazon S3 is written out to SQLite?
Documents, emails and custom fields are exported with mapping, so each mapping is agreed with you first. The firm also decides who may open the archive and how long it is kept.
Other routes
Other routes from Amazon S3 and into SQLite
Where to start
Before and after the move.
Choosing the system comes before a migration. Setting it up and putting AI to work come after.
Choose your systems
Technology stack selection
Decide which practice management system your firm runs, with the evidence and the cost beside it.
About four weeks for a firm of about 15 people. Fixed price, set after a free first call.
Move your data
Data migration
Your data is cleaned up before anything moves, then a trial load is checked against the old system.
Fixed price, agreed after the analysis.
Set up your new system
Implementation
Your new system set up around the way your firm works: configuration, document automation and training.
A small firm of 1 to 10 users typically takes 4 to 8 weeks. A mid-sized firm of 10 to 50 users takes 8 to 16 weeks.
Put AI to work
AI advisory and enablement
A small group of your people learn to use an approved AI tool on real work, under written rules.
Two to four weeks for a small first group. Fixed price, set after a free first call.
Amazon S3 and SQLite are the names and marks of their owners, used here only to identify the two systems.
Moving from Amazon S3 to SQLite?
Tell us what you are moving. You get a plan and a fixed price after the analysis.
We reply within one business day.