Menu
Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1
  • 2

TOPIC: Talend ETL

Talend ETL 5 years 4 months ago #6693

  • astrobl
  • astrobl's Avatar
  • Offline
  • New Member
  • Posts: 2
  • Karma: 0
Hello,

I am trying to upload and import data via the tSugarCRM components in Talend 5.3.
How can I address the AOS Quotes module?

Regards
The administrator has disabled public write access.

Talend ETL 5 years 3 months ago #7224

  • ChrisC
  • ChrisC's Avatar
  • Offline
  • Elite Member
  • Posts: 179
  • Thank you received: 36
  • Karma: 6
astrobl,

Here's a blog post that shows you step by step how to use Talend Open Studio 5.2 to import data into a SugarCRM or SuiteCRM module.

http://www.w-systems.com/blog/sugar-crm/using-talend-to-upload-data-to-sugarcrm-part-2-creating-projects-and-upload

Post back and let us know how it worked for you!
Last Edit: 5 years 3 months ago by ChrisC.
The administrator has disabled public write access.
The following user(s) said Thank You: astrobl

Talend ETL 5 years 3 months ago #7396

  • astrobl
  • astrobl's Avatar
  • Offline
  • New Member
  • Posts: 2
  • Karma: 0
Hello,

Thanks for the link. According to the article, I can reference custom fields in an existing module (Contacts,Accounts,...). But what about the custom module AOS? How do I access that module?

Best Regards
The administrator has disabled public write access.

Talend ETL 4 years 11 months ago #12013

  • nbouwsma
  • nbouwsma's Avatar
  • Offline
  • Premium Member
  • Posts: 159
  • Thank you received: 13
  • Karma: 7
Hey astrobl,

Did you ever get this working? Working with a customer that needs daily large flat-files loaded into a custom module with relationships with two other custom modules. Native sugarcrm import slows way down after around 800 records. I have reached out to cloudbee.com and am currently trying to get their talend advanced sugarcrm component to work. If I get it working I will post the steps taken.
-Nate
The administrator has disabled public write access.

Talend ETL 4 years 11 months ago #12015

  • ChrisC
  • ChrisC's Avatar
  • Offline
  • Elite Member
  • Posts: 179
  • Thank you received: 36
  • Karma: 6
Did you read this thread on talend forum ? "importing csv file into custom SugarCRM module using tSugarCRMOutput"
www.talendforge.org/forum/viewtopic.php?id=29293

They got it working by
editing the xml file of the tSugarCRMOutput component called tSugarCRMCustomOutput_java.xml
in order for the tSugarCRMOutput to identify the custom module of sugar and its fields.

And fixing the "quotes" - because pasting from Microsoft Word inserted "left quote" and "right quote" in the SugarCRM Webservice URL field.
i noticed that the Quotes (") of the "SugarCRM Webservice URL" were invalid.
it turns out i copy pasted the URL from a Word document which caused the quotes to be different than the quotes that i put in when i write the url in the Talend component properties.
i solved it by deleting the quotes and insert them again in the SugarCRM Webservice URL field of the tSugarCRMOutput component (see image2)
The administrator has disabled public write access.

Talend ETL 4 years 11 months ago #12020

  • nbouwsma
  • nbouwsma's Avatar
  • Offline
  • Premium Member
  • Posts: 159
  • Thank you received: 13
  • Karma: 7
Good to hear from you Chris.

Getting data to the custom module works great. It is the data stored in the relationship tables that I am having difficulties with. Along with my primary module there are two additional custom modules that house a one to many relationship to the primary. I am using the advanced sugarcrm component for talend.

Using the web import from the primary module works in the regard but quickly chokes as the number of records climb. I am not constrained by memory or cpu and have tried just about every method of speeding the web input up.

Any further pointers?
-Nate
The administrator has disabled public write access.

Talend ETL 4 years 11 months ago #12023

  • ChrisC
  • ChrisC's Avatar
  • Offline
  • Elite Member
  • Posts: 179
  • Thank you received: 36
  • Karma: 6
Hi Nate,

A couple suggestions to help you!

1. Is your issue similar to this one ?
Importing data which belongs to more than one module.
"Enhancing Importer - allow importing of related module data"
kenneththorman.blogspot.com/2012/06/suga...-importer-allow.html
Same conversation on the sugar forums : forums.sugarcrm.com/f3/solved-importing-...m-6-3-1-patch-80503/

2. If the SuiteCRM import is working, but getting bogged down - and you have unlimited memory and CPU - I suggest increasing PHP memory - (and potentially MySQL memory after consulting with your sys admin) - so as to avoid swapping out to virtual memory on disk.

php.ini
memory_limit = 1024M

or

memory_limit = 2048M

Should make a difference, if the slowdown is really due PHP exhausting available memory...

3., The tSugarCRMAdvOutput Talend component really should be updated to support multi-module import, taking care of the relationships tables, if it doesn't already support this. This definitely needs looking into...
The administrator has disabled public write access.

Talend ETL 4 years 11 months ago #12056

  • ChrisC
  • ChrisC's Avatar
  • Offline
  • Elite Member
  • Posts: 179
  • Thank you received: 36
  • Karma: 6
Continued:

54639 6.5.0 Custom One-to-Many relationship import issue Medium Closed Fixed Defect Import 12/03/2012 18:29
It looks like a bug , having to do with importing custom relationships, was fixed in sugar 6.5.9
What version are you running?

Bug test code : github.com/sugarcrm/sugarcrm_dev/blob/ma...ata/Bug54639Test.php

Two interesting threads about importing custom relationships, on sugar community, that may help you:

community.sugarcrm.com/sugarcrm/topics/d...ule_to_custom_module

community.sugarcrm.com/sugarcrm/topics/c...onships_issues_6_3_0
The administrator has disabled public write access.

Talend ETL 4 years 11 months ago #12127

  • nbouwsma
  • nbouwsma's Avatar
  • Offline
  • Premium Member
  • Posts: 159
  • Thank you received: 13
  • Karma: 7
Hey Chris,

Sorry for the delay, I have a ridiculous amount of resources for Suite via PHP and MySQL config/ini. I am using SuiteCRM 7.1.4. So we are clear the import via module import works great. However when you get up around 100 records the process grinds to a crawl. So to speed things up I am using Talend as it has a better upload rate using a custom talend component create by cloudbee.com.

Where I am am now is that I am able to insert records into custom modules perfectly but the two relationship tables are throwing me for a loop. I have created two jobs. The first job imports data from a CSV to the three appropriate custom modules. The second job uses the same CSV to lookup the previously imported records via name field and returns the UUID for the record. The UUID for the related module record is then concatenated with the related record and placed into a table consisting of the two columns needed to be mapped into the relationship table.

Using the same Talend component or a direct mysql insert the records are never written...
-Nate
The administrator has disabled public write access.

Talend ETL 4 years 11 months ago #12189

  • ChrisC
  • ChrisC's Avatar
  • Offline
  • Elite Member
  • Posts: 179
  • Thank you received: 36
  • Karma: 6
Hi nbouwsma,

Have you run this Talend component multi custom table relationship import issue by Cloudbee's peeterm ? Because it seems like the Talend component by Cloudbee really should handle this import , and if it's not doing it correctly, it should, and they need to know, so they can take a look at it and fix it? Or it may be some obscure setting that needs adjustment for the multi custom table relationship import to work . Either way, if they can help, or can't, please post back and let us know their reply !
The administrator has disabled public write access.

Talend ETL 4 years 10 months ago #12467

  • nbouwsma
  • nbouwsma's Avatar
  • Offline
  • Premium Member
  • Posts: 159
  • Thank you received: 13
  • Karma: 7
Hey Chris,

Yep Peeter over at CloudBee came through- had been working with him during these posts. The job components are up and running nicely. I need to write up a Suite/Sugar Tutorial on using his and Talends components to add relationships between modules. Definitely worth the purchase should anyone else need to migrate data between CRM/ERP/DBs. When I have some free time I will write a tutorial outlining how I am adding 13-9k worth or records daily.

Happy New Year!
-Nate
The administrator has disabled public write access.

Talend ETL 4 years 9 months ago #13627

  • ChrisC
  • ChrisC's Avatar
  • Offline
  • Elite Member
  • Posts: 179
  • Thank you received: 36
  • Karma: 6
nbouwsma wrote:
The job components are up and running nicely.

I need to write up a Suite/Sugar Tutorial on using his and Talends components to add relationships between modules.

When I have some free time I will write a tutorial outlining how I am adding 13-9k worth or records daily.

Hey Nate,

Great to hear the adding relationships between modules is working well with the Talend and Cloudbee components.

Would be great if you could post your Tutorial outlining how you're adding 13-9k worth of records daily ?
The administrator has disabled public write access.

Talend ETL 4 years 7 months ago #16972

  • peeterm
  • peeterm's Avatar
  • Offline
  • New Member
  • Posts: 5
  • Thank you received: 2
  • Karma: 0
Hi everybody,

nbouwsma, good to hear that you could make it work.

By the way here in Cloudbee we have launched recently 15 new components for Talend, now can you can integrate SuiteCRM to even more systems: cloudbee.com
Some examples, you can connect SugarCRM to: ActiveCampaign, BigCommerce, BrainTree, Desk, Erply, FreshBooks, HighRise, MailChimp, PipeDrive, Saasu, Shopify, Toggl, Xero, Zoho CRM, SugarCRM, Magento, Intuit QuickBooks Online.

List will grow even more, we are working on this.

SugarCRM component is also improved and has now all the SugarCRM default schemas built in: cloudbee.com/sugar-crm-talend-component/
If you miss any module or field, you can add it by yourself.

If anybody has some good hints how to make it work even better with SuiteCRM, it would be very interesting.


Regards,

Peeter Mark
cloudbee.com
Cloudbee Integration Connectors
The administrator has disabled public write access.
The following user(s) said Thank You: ChrisC

Talend ETL 4 years 1 day ago #25539

  • booktrunk
  • booktrunk's Avatar
  • Offline
  • Premium Member
  • Posts: 158
  • Thank you received: 2
  • Karma: 0
Did the cloudbee to suite ever work.

I've downloaded the 7 day trial but it only seems to look at accounts and not at any custom fields..... Do I say i'm using version 7 of sugarcrm or one of the older versions for this to work. I've tried 6.4
The administrator has disabled public write access.

Talend ETL 3 years 11 months ago #25727

  • peeterm
  • peeterm's Avatar
  • Offline
  • New Member
  • Posts: 5
  • Thank you received: 2
  • Karma: 0
Hi,

yes, Cloudbee SugarCRM Components Trial supports only Accounts module. However, you can use custom fields. Just modify schema in component settings. You should choose SugarCRM v6.4 in component settings.

I know that you have contacted our support and we sent you all instructions recently.

Peeter
cloudbee.com
The administrator has disabled public write access.
  • Page:
  • 1
  • 2
Time to create page: 0.177 seconds
Powered by Kunena Forum