I am running with the Derby Database and have a separate Live and Test Database.

I want to be able to copy all the data from LIVE into TEST to provide a sensible UAT environment.

I tried just copying the LIVE folder to the TEST folder, just after a Publish, thinking that the data structure would be the same at this moment in time, but when tried to 'Put Under Test' I got an error message saying that an attribute already existed. It was the only attribute added to the version just published. I can't get my head round this.

Can anyone help me to do this please?

  • I am so glad you asked.

    In version 9.1 that is just around the corner we are releasing the Data Migration tool.

    • You will be able to export from production and import into test or the other way around.
    • You will be able to export from one database and import to another database.
    • You will be able to delete existing records before you import the records, or you can ADD imported records to the existing ones
    • You can export/import all objects or selected ones
    • Id's of the records will be automatically taken care of.

    It should be hugely useful. Your comments are welcome.

    We also need some help with testing, so if anyone is interested in testing this new feature please let us know. The more systems we test the better.

Depending on how you set up the BSV, you either established a new (separate) database or are using the standards basdb database. If you did not use a separate database, then your test database tables are all appended with your bsv name i.e., [bsv]_customers. They WILL use this naming convention in the basdbtest database. Let us know on that and then we can better guide you. Personally, I do not know much about the Derby database.

I'd be interested in any tips as to how I could copy a Live database to the Test database when using MariaDB (and using separate databases for both, i.e. not the standard basdb). I thought that maybe I could simply copy all of the tables in the Live database to Test, but I am not sure whether that would create any difficulties, for example because primary keys or sequences would not come along.

This is complex and there is no easy and foolproof way to do this, you have to dabble in the DB and export/import data etc. etc. and if you donΒ΄t know what you are doing, there is a high risk of you messing something up. There is a FR from years back asking for a simple way to do this directly inside Aware e.g with the press of a button and I still think we need something like this.

Thanks Henrik. That is more or less what I suspected and therefore I hadn't started attempting it. Yes, it would be great to have this feature. For example, if you have a client using your system and runs into a particular issue with his database and/or the application (it may not be obvious where the actual cause of the problem is). Ideally you would then make a 'shadow copy' of the database to a test environment to start digging and testing and hopefully find and resolve the problem. In a live database you can't really do tests that alter or add data.

I use the ESF Database Migration Toolkit to pull the Live production data off of a server to my local machine to test and analyse all the time.
It is eaasy once you stop using BASDB and BASDBTEST and have each BS in it's one DB.

10 days later

Just my suggestion - I don't really know much about Derby.

1) create a new BSV in the test DB - be sure to call it the same as the production.
2) EXPORT the bsv from production and IMPORT that into the test environment - that's your start point
3) Publish the test so that it constructs an identical schema in the test
4) export the DATA from the production BSV and then import it into the TEST
5) import the test BSV you've been working on into the TEST environment (if it is different) and immediately PUBLISH it. This should then make any changes required to the schema structure

One thing you may have to do is EXPORT BASIDGEN from BASDB in production and then import this into the same place in the test environment otherwise if/when you start entering new records, you will get duplicate key errors. (and you will need to do this if you later export an up to date copy of the production data. BASIDGEN holds the last ID used by the system

6 days later

I am so glad you asked.

In version 9.1 that is just around the corner we are releasing the Data Migration tool.

  • You will be able to export from production and import into test or the other way around.
  • You will be able to export from one database and import to another database.
  • You will be able to delete existing records before you import the records, or you can ADD imported records to the existing ones
  • You can export/import all objects or selected ones
  • Id's of the records will be automatically taken care of.

It should be hugely useful. Your comments are welcome.

We also need some help with testing, so if anyone is interested in testing this new feature please let us know. The more systems we test the better.

aware_support wrote

I am so glad you asked.

In version 9.1 that is just around the corner we are releasing the Data Migration tool.

  • You will be able to export from production and import into test or the other way around.
  • You will be able to export from one database and import to another database.
  • You will be able to delete existing records before you import the records, or you can ADD imported records to the existing ones
  • You can export/import all objects or selected ones
  • Id's of the records will be automatically taken care of.

It should be hugely useful. Your comments are welcome.

We also need some help with testing, so if anyone is interested in testing this new feature please let us know. The more systems we test the better.

Awesome πŸ™‚ and when you are adding DB management tools inside Aware, why not add an option to export (backup) a DB to an archive file or zipped (or is this possibly already how it will work?) and possibly also have the option to include bsv files etc. (and ideally also make it possible to schedule the export/backup e.g every night + have a matching import feature that restores the DB and bsv again using the archive file). The benefits of this would be obvious for backup purposes but also simplifies migration and things like that.

Awesome πŸ™‚ and when you are adding DB management tools inside Aware, why not add an option to export (backup) a DB to an archive file or zipped (or is this possibly already how it will work?)

Yes, it already works like that - it creates a zip file with database records represented in Aware IM's way - database agnostic.

(and ideally also make it possible to schedule the export/backup e.g every night + have a matching import feature that restores the DB and bsv again using the archive file). The benefits of this would be obvious for backup purposes but also simplifies migration and things like that.

Possibly. But the idea of this feature is primarily data migration, not backup.

aware_support wrote

We also need some help with testing, so if anyone is interested in testing this new feature please let us know. The more systems we test the better.

Sounds great! I am certainly interested in testing this new functionality.

aware_support wrote

Awesome πŸ™‚ and when you are adding DB management tools inside Aware, why not add an option to export (backup) a DB to an archive file or zipped (or is this possibly already how it will work?)

Yes, it already works like that - it creates a zip file with database records represented in Aware IM's way - database agnostic.

(and ideally also make it possible to schedule the export/backup e.g every night + have a matching import feature that restores the DB and bsv again using the archive file). The benefits of this would be obvious for backup purposes but also simplifies migration and things like that.

Possibly. But the idea of this feature is primarily data migration, not backup.

Sure, I understand and makes sense. As Aware still kind of is a RAD platform simplifying a lot of the more complex things, a lot of Aware users are probably not super tech savy and don't have proper backup etc. procedures (if I take me as an example, it took me years of using Aware and several major system crashes with data loss to push myself to learn how to properly have backup procedures and routines in place. Having this built in etc. would simplify life I think for a lot of users.). Having options to do easy backup and restore of DB and bsv (whole app more or less) would in my book also make sense so if nothing else, please add this as a FR for future consideration, thanks.

7 days later

Great feature.

What else is included in new 9.1 and when it will be released?

Thank you

Some of the new features will include parsing objects from a JSON file, improvements to mobile push notifications, improvements to native mobile, improvements to MAC UI and many small changes and fixes

We should release it within a couple of months