Talend ETL

SuiteCRM Forum - English Language Developer Collaboration
1

Hello,

I am trying to upload and import data via the tSugarCRM components in Talend 5.3.
How can I address the AOS Quotes module?

Regards

2

astrobl,

Here’s a blog post that shows you step by step how to use Talend Open Studio 5.2 to import data into a SugarCRM or SuiteCRM module.

http://www.w-systems.com/blog/sugar-crm/using-talend-to-upload-data-to-sugarcrm-part-2-creating-projects-and-upload

Post back and let us know how it worked for you!

1 Like
3

Hello,

Thanks for the link. According to the article, I can reference custom fields in an existing module (Contacts,Accounts,…). But what about the custom module AOS? How do I access that module?

Best Regards

4

Hey astrobl,

Did you ever get this working? Working with a customer that needs daily large flat-files loaded into a custom module with relationships with two other custom modules. Native sugarcrm import slows way down after around 800 records. I have reached out to cloudbee.com and am currently trying to get their talend advanced sugarcrm component to work. If I get it working I will post the steps taken.

5

Did you read this thread on talend forum ? “importing csv file into custom SugarCRM module using tSugarCRMOutput”
http://www.talendforge.org/forum/viewtopic.php?id=29293

They got it working by

[quote]editing the xml file of the tSugarCRMOutput component called tSugarCRMCustomOutput_java.xml
in order for the tSugarCRMOutput to identify the custom module of sugar and its fields.[/quote]

And fixing the “quotes” - because pasting from Microsoft Word inserted “left quote” and “right quote” in the SugarCRM Webservice URL field.

6

Good to hear from you Chris.

Getting data to the custom module works great. It is the data stored in the relationship tables that I am having difficulties with. Along with my primary module there are two additional custom modules that house a one to many relationship to the primary. I am using the advanced sugarcrm component for talend.

Using the web import from the primary module works in the regard but quickly chokes as the number of records climb. I am not constrained by memory or cpu and have tried just about every method of speeding the web input up.

Any further pointers?

7

Hi Nate,

A couple suggestions to help you!

  1. Is your issue similar to this one ?
    Importing data which belongs to more than one module.
    “Enhancing Importer - allow importing of related module data”
    http://kenneththorman.blogspot.com/2012/06/sugarcrm-631-enhancing-importer-allow.html
    Same conversation on the sugar forums : http://forums.sugarcrm.com/f3/solved-importing-related-module-data-sugarcrm-importer-sugarcrm-6-3-1-patch-80503/

  2. If the SuiteCRM import is working, but getting bogged down - and you have unlimited memory and CPU - I suggest increasing PHP memory - (and potentially MySQL memory after consulting with your sys admin) - so as to avoid swapping out to virtual memory on disk.

php.ini
memory_limit = 1024M

or

memory_limit = 2048M

Should make a difference, if the slowdown is really due PHP exhausting available memory…

3., The tSugarCRMAdvOutput Talend component really should be updated to support multi-module import, taking care of the relationships tables, if it doesn’t already support this. This definitely needs looking into…

8

Continued:

54639 6.5.0 Custom One-to-Many relationship import issue Medium Closed Fixed Defect Import 12/03/2012 18:29
It looks like a bug , having to do with importing custom relationships, was fixed in sugar 6.5.9
What version are you running?

Bug test code : https://github.com/sugarcrm/sugarcrm_dev/blob/master/tests/data/Bug54639Test.php

Two interesting threads about importing custom relationships, on sugar community, that may help you:

https://community.sugarcrm.com/sugarcrm/topics/does_anyone_know_how_to_import_many_to_many_relationships_for_custom_module_to_custom_module

https://community.sugarcrm.com/sugarcrm/topics/custom_module_relationships_issues_6_3_0

9

Hey Chris,

Sorry for the delay, I have a ridiculous amount of resources for Suite via PHP and MySQL config/ini. I am using SuiteCRM 7.1.4. So we are clear the import via module import works great. However when you get up around 100 records the process grinds to a crawl. So to speed things up I am using Talend as it has a better upload rate using a custom talend component create by cloudbee.com.

Where I am am now is that I am able to insert records into custom modules perfectly but the two relationship tables are throwing me for a loop. I have created two jobs. The first job imports data from a CSV to the three appropriate custom modules. The second job uses the same CSV to lookup the previously imported records via name field and returns the UUID for the record. The UUID for the related module record is then concatenated with the related record and placed into a table consisting of the two columns needed to be mapped into the relationship table.

Using the same Talend component or a direct mysql insert the records are never written…

10

Hi nbouwsma,

Have you run this Talend component multi custom table relationship import issue by Cloudbee’s peeterm ? Because it seems like the Talend component by Cloudbee really should handle this import , and if it’s not doing it correctly, it should, and they need to know, so they can take a look at it and fix it? Or it may be some obscure setting that needs adjustment for the multi custom table relationship import to work . Either way, if they can help, or can’t, please post back and let us know their reply !

11

Hey Chris,

Yep Peeter over at CloudBee came through- had been working with him during these posts. The job components are up and running nicely. I need to write up a Suite/Sugar Tutorial on using his and Talends components to add relationships between modules. Definitely worth the purchase should anyone else need to migrate data between CRM/ERP/DBs. When I have some free time I will write a tutorial outlining how I am adding 13-9k worth or records daily.

Happy New Year!

12

Hey Nate,

Great to hear the adding relationships between modules is working well with the Talend and Cloudbee components.

Would be great if you could post your Tutorial outlining how you’re adding 13-9k worth of records daily ?

13

Hi everybody,

nbouwsma, good to hear that you could make it work.

By the way here in Cloudbee we have launched recently 15 new components for Talend, now can you can integrate SuiteCRM to even more systems: http://cloudbee.com
Some examples, you can connect SugarCRM to: ActiveCampaign, BigCommerce, BrainTree, Desk, Erply, FreshBooks, HighRise, MailChimp, PipeDrive, Saasu, Shopify, Toggl, Xero, Zoho CRM, SugarCRM, Magento, Intuit QuickBooks Online.

List will grow even more, we are working on this.

SugarCRM component is also improved and has now all the SugarCRM default schemas built in: http://cloudbee.com/sugar-crm-talend-component/
If you miss any module or field, you can add it by yourself.

If anybody has some good hints how to make it work even better with SuiteCRM, it would be very interesting.

Regards,

Peeter Mark
http://cloudbee.com
Cloudbee Integration Connectors

1 Like
14

Did the cloudbee to suite ever work.

I’ve downloaded the 7 day trial but it only seems to look at accounts and not at any custom fields… Do I say i’m using version 7 of sugarcrm or one of the older versions for this to work. I’ve tried 6.4

15

Hi,

yes, Cloudbee SugarCRM Components Trial supports only Accounts module. However, you can use custom fields. Just modify schema in component settings. You should choose SugarCRM v6.4 in component settings.

I know that you have contacted our support and we sent you all instructions recently.

Peeter
http://cloudbee.com

16

Hello,

Does the SugarCRM component for Talend works to import data into SuiteCRM?

Thank you very much.