You can also look into UD6 driver from March Hare. With this driver, IDF is working on XML files, not on DB; XML files that you can directly commit to the trunk. There is thus no more need to sync and all the delete issues are solved. We are currently evaluating this driver, first impression is good.

Hi Andy, thank you for all the explanations how you are doing code management in GIT. Looks pretty mature. So you put a lot of effort in combining UTIMESTAMP comparison of the metadictiionary entities with the GIT. To hold more managing inside the uniface application, you use your own internal entity "to be exported and commited under local GIT". So you avoid the GIT staging completely. As you are using $ude export: - how do you handle the datamodels as $ude expotrs the entire model, do you have means for a finer granularity? And I assume that you do not really use your repository to go back to some older code version. But do you have some ideas in the datamodel area how to handle deleted Fields, Entities ... in that case? Greetings from a cold Frankfurt/Germany morning, Uli

I can't release specific code, but I can describe what I did. All of our developers are working with a shared Uniface repository, so in effect this is the current "master" copy (I use the word loosely here because master generally means something specific in Git). This is significant for handling merge conflicts (see below). We have a single remote Git repo. We started using a local TFS server, but have recently switched to Visual Studio Team Services in the cloud. Every developer has the standard Git client installed and a local Git repo.   Essentially I built a Uniface form (launched from the Additional menu) that presented a combined view of the Uniface repo with the local Git repo and that had a bunch of commands for interacting with them. All interaction with Git was done through the command line and COMMAND() operations. It is slightly complicated because of the Git (bash) shell, so I ended up having to create a shell file with the git commands I wanted to run and then running the (DOS) command to launch git and run that file. If any output was needed from the git command then that was piped into a text file that was loaded in when control returned to Uniface. Kind of convoluted but with a bit of common code to manage temp files it all worked out. When a developer ran the UI for the first time then it would initialise their local git environment and repo from some ASN file settings. So other than installing Git, the developer did not have to do anything specific. That initialisation will clone any remote git repo and checkout the develop (or whatever you call it) branch. Within the form, a developer could create a branch or switch to another one and under the covers the appropriate git checkout was run. From that point on, the form showed the Uniface items that had been modified since the modification date of the corresponding item in the git workspace (i.e. the current branch). Through our naming conventions it excluded items not meant to be under source control. The form used a tree widget so that you could just check on certain types of items at a time. The developer could then choose to "stage" an item(s), which didn't actually do much other than record the name of the item in another place on the form. The ultimately goal for a developer would be to drive the list of changed items down to 0.  Once they had staged all the items they wanted then they could "commit" them (along with a commit message) and this did the actual work. Each item is exported (via $ude(export) as xml) into the local git workspace, followed by a git commit command. So in git terms we did not use the staging area and changes in the workspace area are immediately committed, which again simplifies merge conflicts. When the developer was happy with their commits then they could be pushed up to the remote git repo. I didn't write anything to generate a pull request as that was pretty easy within TFS. Similarly the merging of individual branches to the main develop/master branches was left as a TFS operation.   The big difference between Git and other source control systems is that it is always dealing with the entire code base. Every commit is a snapshot at a point in time - now Git is efficient in piecing together what has actually changed so it isn't actually throwing every single file around all the time. But it is a different mindset. And part of the philosophy is that you have to handle merge conflicts. Different developers are only going to export/commit the components that they've changed, so if devA has committed comp1 and devB has modified comp2, then devB's commit really needs to include that latest comp1. So when our Git UI starts up, it automatically pulls the head of the develop branch from the remote git repo (develop was the name of our main branch, and the branch that individual branches were merged into once they had completed code review). The user could not directly interact with the develop branch locally through the UI so this pull is safe, but the latest changes need to be merged into whatever branch the user is currently working on. Typically this is a fairly easy process handled by the git merge command and just results into another commit to wrap things up. Part of my Unifac service handled it all automatically. But there will be cases where there are merge conflicts because of differences between the current branch and the develop branch. Traditionally these differences would be handled via a visual merge tool by a developer going through them line by line. That is a little tricky in Uniface exports because changes are not so concisely expressed or always in terms that a developer usually sees. But the solution is actually pretty simple given that the real master version is always in the common Uniface repo whereas a traditional developer is working in their git workspace directly. Therefore my tool checks each conflict and if it is because of a changed version in the local git workspace then that means the developer is working on it (they've done a commit with it) and so we just need to reexport it from Uniface, otherwise just checkout the version from the develop branch because the version in the current branch is old.   There is a lot of power with the git commands, particularly log that allows you to get all kinds of details about the state of the repo. Much of the functionality revolved around parsing results from that command. One other nice thing during development of the interface was that I could have a git shell open in a separate window and try things out interactively beforehand. I also built tools to use the git diff and blame commands to allow a developer to explore the entire history of an item (actually that reminds me to go raise a wishlist item on the html widget!). Similarly we could leverage the git reporting options to determine what has actually changed so that the build process doesn't have to import the entire code base, which reduced our build times significantly. Now obviously you don't need to go as far as I did in making it easy and pretty for developers! All you really need from Uniface is a way to export an item into a text file and place it into the git workspace, and of course there is a standard UDE screen for that. Everything else can be done from the git shell or other git tool such as Atlassian's Source Tree.  Hope this helps.

Hi Andy, Right now I check what it takes to transfer the dITo uniface-subversion bridge from 2009 to GIT. Therefore I'm interested in your implementation of GIT commands as well. How can I get more infos about it?   Greetings from Frankfurt/Germany, Uli

Andy Heydon said Hi, This past year I built an interface from a Uniface 9 installation to Git, and that used TFS as the remote store. The only think we installed on the desktop was the standard Git client. Most of the interaction from the UDE was using the Git command line via "OS".Command() to a local Git repo, but I built a nice UI that hide the gory details of the various Git commands from the developers. That included functions to create/delete branches, do commits, push to the remote, compare versions (either between local Git repo versions and/or the UDE version, and against the original template), and show the blame report. In our environment the UDE repo was shared by all developers so that was ultimately the "master" version, but this made merge conflicts easier and it also meant that we skipped the concept of staging. From the UDE, a developer exports and commits a component in one step, and therefore it is simple to change branches without being concerned about uncommitted changes.

Andy,   I am very interested to hear your solution to this.  Do you have more details? thanks, Ben  

Hi Benjamin,   There is a video about UD6 and Version control at: http://theu.info/#video-112 titled "introduction to version control" some 5 minutes long. As you can use any command-line based Version Control Tool, GIT should be easily incorporated.

you can reach me at gardenspider63@msn.com

Can you provide some way to communicate outside of this forum, because the answers will be quite lengthy (and perhaps confidential)?

Andy Heydon said I can't release specific code, but I can describe what I did. All of our developers are working with a shared Uniface repository, so in effect this is the current "master" copy (I use the word loosely here because master generally means something specific in Git). This is significant for handling merge conflicts (see below). We have a single remote Git repo. We started using a local TFS server, but have recently switched to Visual Studio Team Services in the cloud. Every developer has the standard Git client installed and a local Git repo.   Essentially I built a Uniface form (launched from the Additional menu) that presented a combined view of the Uniface repo with the local Git repo and that had a bunch of commands for interacting with them. All interaction with Git was done through the command line and COMMAND() operations. It is slightly complicated because of the Git (bash) shell, so I ended up having to create a shell file with the git commands I wanted to run and then running the (DOS) command to launch git and run that file. If any output was needed from the git command then that was piped into a text file that was loaded in when control returned to Uniface. Kind of convoluted but with a bit of common code to manage temp files it all worked out. When a developer ran the UI for the first time then it would initialise their local git environment and repo from some ASN file settings. So other than installing Git, the developer did not have to do anything specific. That initialisation will clone any remote git repo and checkout the develop (or whatever you call it) branch. Within the form, a developer could create a branch or switch to another one and under the covers the appropriate git checkout was run. From that point on, the form showed the Uniface items that had been modified since the modification date of the corresponding item in the git workspace (i.e. the current branch). Through our naming conventions it excluded items not meant to be under source control. The form used a tree widget so that you could just check on certain types of items at a time. The developer could then choose to "stage" an item(s), which didn't actually do much other than record the name of the item in another place on the form. The ultimately goal for a developer would be to drive the list of changed items down to 0.  Once they had staged all the items they wanted then they could "commit" them (along with a commit message) and this did the actual work. Each item is exported (via $ude(export) as xml) into the local git workspace, followed by a git commit command. So in git terms we did not use the staging area and changes in the workspace area are immediately committed, which again simplifies merge conflicts. When the developer was happy with their commits then they could be pushed up to the remote git repo. I didn't write anything to generate a pull request as that was pretty easy within TFS. Similarly the merging of individual branches to the main develop/master branches was left as a TFS operation.   The big difference between Git and other source control systems is that it is always dealing with the entire code base. Every commit is a snapshot at a point in time - now Git is efficient in piecing together what has actually changed so it isn't actually throwing every single file around all the time. But it is a different mindset. And part of the philosophy is that you have to handle merge conflicts. Different developers are only going to export/commit the components that they've changed, so if devA has committed comp1 and devB has modified comp2, then devB's commit really needs to include that latest comp1. So when our Git UI starts up, it automatically pulls the head of the develop branch from the remote git repo (develop was the name of our main branch, and the branch that individual branches were merged into once they had completed code review). The user could not directly interact with the develop branch locally through the UI so this pull is safe, but the latest changes need to be merged into whatever branch the user is currently working on. Typically this is a fairly easy process handled by the git merge command and just results into another commit to wrap things up. Part of my Unifac service handled it all automatically. But there will be cases where there are merge conflicts because of differences between the current branch and the develop branch. Traditionally these differences would be handled via a visual merge tool by a developer going through them line by line. That is a little tricky in Uniface exports because changes are not so concisely expressed or always in terms that a developer usually sees. But the solution is actually pretty simple given that the real master version is always in the common Uniface repo whereas a traditional developer is working in their git workspace directly. Therefore my tool checks each conflict and if it is because of a changed version in the local git workspace then that means the developer is working on it (they've done a commit with it) and so we just need to reexport it from Uniface, otherwise just checkout the version from the develop branch because the version in the current branch is old.   There is a lot of power with the git commands, particularly log that allows you to get all kinds of details about the state of the repo. Much of the functionality revolved around parsing results from that command. One other nice thing during development of the interface was that I could have a git shell open in a separate window and try things out interactively beforehand. I also built tools to use the git diff and blame commands to allow a developer to explore the entire history of an item (actually that reminds me to go raise a wishlist item on the html widget!). Similarly we could leverage the git reporting options to determine what has actually changed so that the build process doesn't have to import the entire code base, which reduced our build times significantly. Now obviously you don't need to go as far as I did in making it easy and pretty for developers! All you really need from Uniface is a way to export an item into a text file and place it into the git workspace, and of course there is a standard UDE screen for that. Everything else can be done from the git shell or other git tool such as Atlassian's Source Tree.  Hope this helps.  

Hi Andy, I try to execute git commands from Uniface IDF. You mentioned: "All interaction with Git was done through the command line and COMMAND() operations. It is slightly complicated because of the Git (bash) shell, so I ended up having to create a shell file with the git commands I wanted to run and then running the (DOS) command to launch git and run that file. If any output was needed from the git command then that was piped into a text file that was loaded in when control returned to Uniface. Kind of convoluted but with a bit of common code to manage temp files it all worked out." If I try to execute some common which doesn't need a repo, it is working fine, for example git --version But if I try to execute git commands such as add/commit/push it doesn't work. Can you explain more about how you handled these functions (add/commit/push) after exporting the components from interface. Note : Once I exported the component from the IDF interface the files will be overwritten in the repo directory in windows. I use tortoiseproc.exe in COMMAND operations for SVN repo to execute the commit command. But I don't find any equivalent in git. Thanks, Satheesh

Andy, thanks for all the details how you use naming conventions and many small datamodels as a your base for the code management. Very interesting to get real life experiences "from the trenches", thanks for sharing, Uli

Uli, Currently each model is its own configurable item. We have lots of models so that helps to split things up a bit, but it would be nicer to make it more granular. If we wanted to go to a lower level, e.g. table then we could use $ude copy and some where clauses on the UC* entities. Shouldn't be that tricky, but as our previous source control was at the model level and that is what our build system is set up for, then we kept the same level. You are correct in that we aren't making any attempt to rollback changes. It is a situation that very rarely happens and so we haven't made any special arrangements for it. I did build comprehensive diff tools though so that if a change doesn't work out then we can easily see what we have to move to code back. Given that commits in Git are cheap then it is simple to construct a timeline of changes. Andy

We have a LOT of Uniface developers using our database driver to store your Uniface source code in text files: http://march-hare.com/library/html/ud6blurb.htm   Once your source code is in text files, managing them with Git is of course just like managing anything else in Git. There are options in UD6 so that UTIMESTAMP/UCOMPSTAMP can be 'stripped out' of the 'source code' and UD6 reads the 'timestamp' of the 'file' (and .frm 'file') to supply the information to the IDF when it is read back in. If you have any questions about UD6 specifically, please let me know via sales@march-hare.com   A lot of customers also create an 'additional menu' so that the Git actions can be controlled directly from within the IDF.  Of course not only Git is supported, we have customers using CVS, Perforce, ClearCase, PVCS, SVN and more.  The real advantage of using UD6 is that there is only one copy of your source code: the file, you are not importing/exporting form the database and needing to manage the possibility of getting them out of sync.  Once your source code is in text files there are also many other things you can do, like search/replace in text files, and build management (Jenkins, Bamboo etc.).   We have a lot of videos and customer stories here: http://theu.info/   I'm visiting some sales leads in Europe and USA in March - so if you would like a complimentary on site demonstration for your company or your user group, please let me know.  I'll be doing it again in May probably, if that timing works better for you.

Very Interesting ideas for this issue.  I'm still trying to come up with a solution to using GIT for our development version control.  From what I gather in this discussion, integrating GIT into Uniface directly is not possible, but I'd like to hear more about everyone's solutions so that I can more effectively develop my own solution that meets our needs best.   I'm admit, I am a "newbie" when it comes to $UDE techniques, but it does look promising the more I look into it.  I've been using Uniface for more than 20 years now, in various roles and capacities, and I am always interested in expanding my knowledge and experience with the product.  I had been considering using Microsoft Visual Source Safe as an alternate for version control, but this is no longer available from Microsoft.  On their website, MS recommends either GIT or Team Foundation Server, neither of which I am that familiar with.  (Off topic- Does anyone remember UVCS?) The current source code control interface in Uniface IDF was designed to be used with SCC compliant version control programs, which GIT and TFS are not.  Also, the techniques must be different for versioning, especially in a multi-developer environment.  There is no longer form locking, but "threads" instead.  Without  a direct interface, how do manage you files with GIT?  How do you backup everything the first time around and what file format do you use?  From that point on, how do you manage the project to project versioning tasks? Thanks in advance for all your input!

ErikFiets said We use the Uniface repository and $ude("export", ...) to export components, include procs and the model. Unfortunately you cannot export dtd? Then you can use any version control software you like.

For DTD's I use the Proc statement entitycopy instead of $ude.

Iain Sharp said What I don't get with all this is the re-integration from version control. Things like the model, when imported, do not delete pre-existing fields if they have been dropped from the entity. (Ditto indexes etc.)

Hi Iain, the trick is to use a CIF file and have a little routine which appends a "cleanup xxx" line at the end in the creation process. Loading this CIF file will delete anything in the repository for this object which is not mentioned in this CIF file.   See: http://unifaceinfo.com/forum/ditodoitourself/importexport-and-deleted-items-in-the-datamodel/

What I don't get with all this is the re-integration from version control. Things like the model, when imported, do not delete pre-existing fields if they have been dropped from the entity. (Ditto indexes etc.) Does this mean that to re-integrate to a particular version, you start with a blank IDF and reimport everything? Does that not take forever? 

We use the Uniface repository and $ude("export", ...) to export components, include procs and the model. Unfortunately you cannot export dtd? Then you can use any version control software you like.

It would really make no difference if you are in the Uniface IDE or not. I've regularly used the Git bash shell and Uniface initiated Git commands concurrently. Git commands are performed against a set of files, and the UDE goes against a database. You need to use the UDE to export components to those files, so you might as well use the standard export function or build your own function based around $ude() and hang it off the Additional menu. Unless you are doing something fancy with each developer having their own Uniface repository then all traffic is going to be one way (UDE->Git). Our environment required commit messages to be in a specific format, so it was better to build our own functions to enforce that. We could also automatically handle all of the Git merge conflicts without a developer having to be concerned with all the Git commands.

Might be better to stay out of the Uniface IDE when executing git commands.  How about using msysgit (for windows) with the bash shell - works well for us - https://git-for-windows.github.io/ Or if you prefer a GUI, then Attlassian have a free tool called SourceTree.

ulrich-merkel said There were attempts to provide an interface with SUBVERSION by some extra items in the ADDITIONAL menu.

There is a solution available with subversion, it's offered as part of a Uniface professional services offering. 

Presumably the web portal has a HTTP based API so you could use UHTTP to communicate with it.

We are using GIT, but not TFS(Team Foundation Server?) We don't have a GIT desktop client, but use a web portal to go to GIT repositories. I'd like to set something up between IDF and the portal.  Do I need to make a custom Uniface form to accomplish this using an OCx control? As I said, I tried using PUSHOK(for GIT), but so far it will not work.

We created a uniface form with a .Net ocx for linking it to TFS source control (and with TFS 2015 supports GIT commands, but we use native TFS). The form we added in the additional menu. We use the $export functions from Uniface to create the XML files and these files are in TFS. Also with listing files we created ourself. (reading DICT tables and dumping lines in files)   On client side we install the .Net OCX library containing all the necessary .Net TFS dll's The only problem we have is that the checked out form / service is not locked to one user. And other users can change in the checkout of someone else. This because we have one central UDE repo. We are looking to use UD6 (http://march-hare.com/), this allows a repository on files based and no database. And then we have file locking.

Hi, This past year I built an interface from a Uniface 9 installation to Git, and that used TFS as the remote store. The only think we installed on the desktop was the standard Git client. Most of the interaction from the UDE was using the Git command line via "OS".Command() to a local Git repo, but I built a nice UI that hide the gory details of the various Git commands from the developers. That included functions to create/delete branches, do commits, push to the remote, compare versions (either between local Git repo versions and/or the UDE version, and against the original template), and show the blame report. In our environment the UDE repo was shared by all developers so that was ultimately the "master" version, but this made merge conflicts easier and it also meant that we skipped the concept of staging. From the UDE, a developer exports and commits a component in one step, and therefore it is simple to change branches without being concerned about uncommitted changes.

There were attempts to provide an interface with SUBVERSION by some extra items in the ADDITIONAL menu.