Lost all my custom dropdowns between 7.6 and 7.8

Good day.

I’ve spent a solid week now on this and I think I have finally isolated my issues quite well.

First, the history: I tried to upgrade my suitecrm 7.6.6 to 7.10.13.

I used the upgrade wizard. It got me up to 7.10.13 (that’s what ‘about’ says)

However, as you can see from this screenshot, the custom drop downs and custom fields somehow do not display correctly. However, as you can see, there is clearly a reference to them in the database somewhere because I was able to see them in the Studio’s layout editor and was able to lay them out… No matter what I tried (I tried every rebuild and repair I could think of) it seems that it would not connect all my dropdowns and other custom fields.

In the Production Instance (running: 7.6.6) I can see all my custom labels and their drop downs. When I go into Studio/Dropdown Editor, everything is there perfectly and works great.

Then, just to test, I did a clone of this setup with a cloned suitecrm directory but a brand new database. I imported the database (DB) from the Production Instance, into this Test Instance DB.

Everything works perfectly in the Test Instance, exactly as it does in the Production, so, suitecrm, in this version (7.6.6), and the database, etc, are working just fine with all my custom stuff.

The only thing that I can think of now is that there is something preventing my custom work in 7.6.6 from working properly in 7.10.13.

Maybe theme?
Maybe permissions?

Any help at all sincerely appreciated!

Custom dropdowns are not stored in the DB.

They are in php language files (one per language) found in the following folder:

custom/include/language

If you have run an upgrade I believe that SuiteCRM doesn’t delete anything just in case there could have been a conflict with certain files.

To find your original language files that store your dropdowns you should go to the folder:

upload/upgrades/patch/<the patch you applied>/custom/include/language

Then you have to compare the new file with the old one and merge them manually into the new one.

Before modifying any of these files I highly recommend that you take a copy!

you are some kind of hero!!

i had nearly zero hope of anyone understanding my situation but this seems close. Unfortunately, it seems that not everything is perfectly as you say in my file structure, but I know for sure is this:

  • the original crm install is working perfectly in 7.6.6 (and still is)

  • i have full access to these files in the cpanel directory.

First, this is the list of my files in the custom/include/language of the working instance (7.6.6)
(see appropriately named screenshot filename)

Then, I have this in my /patch/include/language
(see screenshot)

As you can see, they are different.

So, if i understand you correctly, I should copy the contents of the ‘patch’ directory of this instance (7.6.6) into the /custom/include/language/ of the new instance (7.10.13) ?

Your help is sincerely appreciated by the way!!

I just wanted to post a quick update here since things are now getting a bit desperate for me (urgent) and I wanted to make sure others can benefit with me as I become the guinea pig on this thread (I coudn’t find another thread like it…)

While waiting for a response I decided to compare and contrast the files and their content using the ‘view’ option in Cpanel. I discovered that the missing items that I need in my new Test Instance, are indeed found in the folder custom/include/language as described by @amariussi below.

In my case, all of the missing items appear to be in the first file in the list called ‘en_us.lang.php’

The other files appear to be used in the core of Suite CRM so I’m going to attempt to work with this file and bring the needed content into the new instance.

Because the word ‘merge’ was used by @amarussi, I decided not to just copy and paste the old file above into the new directory because I assumed the new file would contain critical code that would break everything if I did this.

So, I’m going to interpret the words ‘merge them manually’ to mean, copy and paste the items from my old ‘en_us.lang.php’ file into the new one in 7.10.13, line by line until the new php file contains the missing items from the old one.

Wish me luck and here we go… :ohmy:

Well, I have my first progress here so I thought I would share this as well since I know absolutely nothing about PHP and I have to figure this out now. It took me a while but the PHP of my Test Instance (7.6.6) in the file mentioned above has this example in this format:

);
$GLOBALS['app_list_strings']['prod_stage_list']=array (
  '' => '',
  'ord_cancel' => 'ORD - CANCELLED',
  'ord_pend' => 'ORD - PENDING',
  'ord_in_prod' => 'ORD - IN PRODUCTION',
  'ord_produced' => 'ORD - PRODUCED - NEEDS BOL',
  'ord_produced_wait_pu' => 'ORD - PRODUCED - BOL SENT AWAIT PU',
  'ord_shipped' => 'ORD - SHIPPED',
  'ord_received' => 'ORD - RECEIVED (closed)',
);

So, I chose a part to start pasting in the Test Instance version of the same file since it’s completely different from the old one (I was right you can’t just copy/paste the whole file in!).

After studying the new and working formats in this good and few file, I finally landed on this format using this real example from our old customized drop-down:

 //waynes test of a merged dropdown list
    'prod_stage_list' => array(
        '' => '',
        'ord_cancel' => 'ORD - CANCELLED',
        'ord_pend' => 'ORD - PENDING',
        'ord_in_prod' => 'ORD - IN PRODUCTION',
        'ord_produced' => 'ORD - PRODUCED - NEEDS BOL',
        'ord_produced_wait_pu' => 'ORD - PRODUCED - BOL SENT AWAIT PU',
        'ord_shipped' => 'ORD - SHIPPED',
        'ord_received' => 'ORD - RECEIVED (closed)',

    ),

Then, after I saved it, I went in, had to give it a new ‘display label’ in Studio / quotes / fields.

Note: if your format is not EXACTLY the same as this example above, it will not only not work but you will not be even able to log into your suitecrm instance. It was pretty cool to learn that one stupid missing space could kill an entire company. I’m teaching my 6 and 10 year old daughters this today!

Then, I went in to Studio / quotes / layouts / edit view and located it and threw it in as a test in the layout and saved it.

Boom! It showed up and seems to be working. I opened things up, edited a test quote and saw my old custom field appear as it should and with a drop down.

The next thing I did was save something to see if it was actually working and saving stuff to the database. I changed the order stage in this drop down, saved it, logged out, logged back in, went back to the record and it was still there and saved. I now assume that the database also is working.

It would be extremely cool if there was a feature to ‘import custom fields in’ somehow. This is downright scary and hard for me. HA. But hope this post is helping someone other than me while I figure it out.

I’ve backed up my 7.10.13 php file here so that worst case at least I have the original un-edited version to reflect on / reference.

Also as part of the process, any time I update the file, I save my changes and also refresh suite crm to see if I broke anything with the last change. Good thing because a few times I did!

I have now done a few tests. It appears I will also have to manually merge the custom additions I made to the core Suitecrm fields (ie. taxes, sales stages, etc).

This will be painstaking but I think I know what to do now.

Just for the records and if anyone is reading this, it would be great to know if my next planned steps look good. Here is what I plan for this week:

  1. Rebuild all the lists/dropdowns custom fields (as per this forum thread)

  2. Backup the database (current Production) one last time since we have been using it since my last backup

  3. Do a completely fresh install of 7.10.13 LTS on the server including a fresh database, just in case I messed anything up during step 1

  4. Import the current productioin database into the new DB created in step 3

  5. Connect suite crm to newly imported database via config.php

  6. Paste the merged updates created in step 1 (custom fields) into language file

  7. Rebuild all the layouts of modules with built in suitecrm tools matching old suitecrm layouts

  8. launch it as live production site after i’ve confirmed all the layouts are working and each field is functioning.

Thanks

Please attach the “old” and the new files here. Probably the Forum won’t let you add .php files so change the extensions to .old and .new and, if necessary put them in a .zip file.

If you can’t attach the files just past the full contents in two separate code tags:
(the one surrounded by a red box)

To be honest for this operation you don’t really need to take a full back-up, however it is always better to do so.

After you have taken the back-up you can proceed with just replacing the merged file that I will provide you (I can do it in a faster way since I know php!).

If this doesn’t work then we will look at other possible solutions,

(second time uploading wrong php, sorry :frowning: )

Thanks @amariussi

I have made some // comments in the .new file, so you can see where i started pasting the updates and fixing format. My next step was to do a ‘find and replace’ on those strings with the old php and replace with new format perhaps. If that is your plan I can probably figure that out, however, who knows if my final efforts will work. I will keep pushing forward doing what I’m doing because i’m desperate but all your efforts will not be wasted because this thread will be super valuable for others. Surely I am not the only one this will happen to and i think these kind of threads make Suitecrm super awesome. That’s why I’ve been here forever!

Thanks for your help!

Update: it looks like I was pasting my edits into /include/language php file in the Test Instance, instead of custom/include/language

The reason this happened is that there was no file in there. Probably because I had not created a custom field in the new installation…

So, I will create one field right now to see if it generates the custom file in the right spot… then I can upload both, but right now I just have access to the old one.

Update again: it seems that in 7.10.13 that these custom fields are not stored in /custom/include/language

I created a fresh ‘test field’ in studio in quotes, saved it, customized field appeared in list as it should, but when I went to custom/include/language there was no .lang file generated.

Maybe I can do a search for .lang and find it…

hmm… i found something here: /custom/modules/AOS_Quotes/language/en_us.lang.php

Dang… here it is…

so I realize now I have to move my old php content into here, not into the main language file otherwise probabably that won’t get up graded in the future

The good news is that formatting appears ‘more similar’ in this file compared to the one I was pasting into.

Fun times! Good learning!

Just to test my theory, I went into the /custom/modules/Opportunities/ directory to find the language php file. It was not there. Neither was a /language directory

Then, I went into the Opportunity Module in the Test Instance, into Studio, and created one single field called ‘test opportunity field’.

Then I went back to /custom/modules/Opportunities/ directory and refreshed and sure enough the /language directory had appeared and inside of it was the en_us.lang.php with my test field.

So, now I have to go back and create a test field in each module that I have customized and paste over the customized code from ‘old’ into ‘new’. The hard part will be that Suitecrm 7.6.6 had everything in one file so now I have to break it out. But, looking promising…

EDIT: Well, well. After I created a brand new test drop-down, just like the experience above, a new en_us.lang.php file was created in this directory:

custom/include/language

This explains why I couldn’t find it before :frowning:

Now I know where drop-down lists are stored in suitecrm 7.10.13 :slight_smile:


Original message:

Sorry for long dialogue but i’m hoping the SEO benefits on this text will help others long term. I now realize that my customized drop-downs are not showing up where they used to be in the drop-down editor section of Studio. Do you have any idea idea where this data is stored in 7.6.6 compared to 7.10.13? It looks like I might have to manually move this stuff too before beginning my layout rebuild and all the stuff above…

I am getting a little confused.

From your latest post you don’t have any custom dropdown. Do I understand well?

The AOS_Quotes dropdowns should not be edited by hand. If you want to do it you should do it in Studio.

About the two notations that you refer.
In php you can write an array in varios ways and it means the same. So the two notations that you refer to are exactly the same. It’s just a different way to write the same thing like in the following example:


$a = array();

$a['aaa'] = array();

$a['aaa']['bbb001'] = 'j';
$a['aaa']['bbb002'] = 'k';
$a['aaa']['bbb003'] = 'l';

is the same as:


$a = array(
          'aaa' => array(
                           'bbb001' => 'j',
                           'bbb002' => 'k',
                           'bbb003' => 'l',
                        ),
         );

which is the same as:


$a = array('aaa' => array('bbb001' => 'j', 'bbb002' => 'k', 'bbb003' => 'l',),);

Please try to explain again what has happened because I think that I may have missed something.

Thanks again.

In summary, in the first post, I thought I was pasting into /custom/include/language/en_us.lang.php.
In fact, I was pasting into /include/language/en_us.lang.php (which appeared to be a system file and a dangerous one to edit.

Then I went to :/custom/include/language/ and discovered there was no php language file.

Then I went into Studio, created a fake dropdown, and boom. The language file appeared in
/custom/include/language/

Then I went into that newly-created php file in this directory and observed the formatting of the code and noted that it is exactly the same as my original 7.6.6 language file.

I copied the contents of the old one, into the new one and boom. It seems like everything is working.

Now I’m just editing the field names (which got lost along the way) and re-making the layouts.

Everything seems to be working.

I will update my status and your teaching here is also much appreciated.

For anyone following along, what I’m doing now is re-mapping the ‘display name’ of each field which got lost along the way. (see screenshot)

Although there may be a better way if you’re a programmer, what I’ve done is copy/paste the old list from studio / opportunity / field list and paste into a spreadsheet. Then i just do a control F in the doc and work through each one one by one re-naming each field. There is probably a file somewhere that would let me do this but, alas, forward motion is the name of the game.

I’ve also realized that you have to do this for each and every module you have customized fields for.

  1. go to studio / module / fields
  2. copy paste list into spreadsheet
  3. go into new instance, rename each field
  4. repeat.

I’ll keep my annoying journal going here…

sadly this all ended bad. Spent 14 hours rebuilding all the stuff (see this thread) but then only thing left was cron jobs / email. In the process, I hit the ‘repair/rebuild’ feature and boom. Can no longer log into CRM (probably it changed PHP to what it should be and killed everything - white page of death) :frowning:

Anyway, I’m going to try again but this time with a fresh install from website (other one, I think, was output of an attempted and failed upgrade wizard verseion)

Hope this thread still has value, ha.

Here I am again today. Today I tried to do a simple ‘move’ of the whole /custom/ directory over to the new install.

Unfortunately it didn’t grab the mappings of the fields, but, as you can see ‘the rest’ seems to be working which is somewhat encouraging. I think the problem is that you have to do this all in studio to make it ‘stick’. If there is a way to ‘re-map’ what I have here, do let me know! I’m also going to hit the repair/rebuild button now before I go to far to see if it breaks my system again. If it does, we know something is broken anyway…

…broken! dang it. so something that I did yesterday in the custom fields inherently breaks the PHP :frowning:

just to test the theory, i renamed the ‘newly fixed’ /custom directory back to ‘custom_dead’ and then I renamed the old and originally installed custom folder back to ‘custom’ and refreshed and boom. it’s alive again. so definitley something is broken inside my custom folder.

The question is, how do I move my old custom settings from 7.6.6 custom directory over to 7.10.13?

Since I’m such a renegade, I think I’m just going to do something stupid and copy the 7.6.6 custom directory over to the 7.10.13 install to see how badly it breaks for the team… this thread is becoming some kind of hackers festival…

…what the heck? It seems to work! It’s like my old 7.6.6. layouts are now working in 7.10.13 install… let me try to break a good thing again. Going to hit the repair and rebuild button…

…didn’t break. what the …

… now to test logging in and out… good!

… now to test updating a custom field in a module…

This is amazing. It seems that it’s all working now. If anyone reading this sees any future pitfall for this method, let me know, otherwise, I’ll leave this note saying this:

If you have accidentally messed up an update, or your update wizard won’t work, it seems that you can just copy the /custom directory to your new install and leave the rest and it ‘just works’

I’ll report back here if I find any headaches, otherwise, hope this helps someone else.

@wayneoutthere I have the impression that you are getting carried away and you are doing things that you shouldn’t do without knowing PHP and the SuiteCRM architecture.

I believe that it would be better that you deal with one issue at a time and, before doing anything, you obtain advise towards a diagnosis of the problems of your SuiteCRM instance and, only after this diagnose, you should operate the necessary changes and fixes.

Obviously you need a full back-up before you start doing anything.

Initially I recommended that you took a back-up and edit just one file by merging two different files after having saved copies of both.

I wanted to see the result of this modification.

If that did not work I would have asked you for what I normally ask: check all the logs to try and find if something is wrong.

Then, since you kept on modifying things, essentially without sufficient knowledge, I asked you to stop and to try to explain again the whole issue that you have because I am not sure I have understood it.

Fortunately, since you say that you are starting from a fresh install, I believe that you don’t have any or too much precious indormation/data/personalisations.

If you want, and have a little patience (because the Forum has long empty periods between one communication and the other) we can try to understand what is wrong in your instance of SuiteCRM and correct it.

I appreciate it sincerely but I couldn’t wait because my business runs on this and i had orders to get out the door. I am not surprised that you considered me carried away. I felt much worse than that.

I don’t know what went wrong with the original install but the current set up seems now ‘mostly right’.

I did the rebuild and repair and it still seems function as mentioned.

The only items left are:

  1. cron jobs are showing links to the old directory name I gave it while I was testing (need to fix that)

  2. a job is stuck in ‘running’ under workflow in scheduler

That’s about it. Rest should be ok.

sorry for being a renegade. I know how forums work so i figured it was better to keep the dialogue going in case myself or someone else gets into any kind of similar situation.

bonus: i learned a little PHP and a little scrm architecture :slight_smile:

Another good update here:

  1. the stuck cron job got ‘freed’.

  2. the link in the crm-generated emails is now correctly pointing to the crm after I changed name of directory.

For number 2, I did a control + F in the config.php file and found the site_url entry was the culprit. It had not changed when I renamed my directory in cpanel. I manually changed it to correct location and it works perfectly.

For number 1, I am not sure what freed the stuck job, but I did these steps:

  1. changed the ‘clean job cue’ to 1 minute in the scheduler

  2. changed the cron tab on my shared host from:

cd /home/myusername/public_html/mydomain.ca/crm; php -f cron.php > /dev/null 2>&1

to:
/usr/local/bin/php -q /home/myusername/public_html/mydomain.ca/crm/cron.php > /dev/null 2>&1[/quote]

I wanted to point out another thing here that might have some value in the project. It says this at the bottom of the scheduler this:

[quote]In order to run SuiteCRM Schedulers, edit your web server user’s crontab file with this command:
sudo crontab -e -u greatcawa
… and add the following line to the crontab file:

          • cd /home/greatcawa/public_html/mygcw.ca/crm; php -f cron.php > /dev/null 2>&1 
            

I was thinking that since Suitecrm is distributed through Softaculous in cpanel (that’s how I found it!) that it would be good to add this comment:

“If you are using suitecrm on a shared hosting environment on cpanel, you will need to add your cron entry there. Note that you do not have root access so the above command will not work for you. Talk to your hosting company tech support or search our forums for some cron configurations that worked for them”

Thoughts?

Thanks!! :cheer:

I mean to write some better tutorials for all these configurations, based on my experience helping people in the forums. It’s hard to fit everything into a screen inside the app, though. I think it will have to be in the online Documentation, and then we can include a link in that screen.