Menu
SuiteCRM dedicated support by SalesAgility
Welcome, Guest
Username: Password: Remember me

TOPIC: Perform Lucene Index doesn't work

Perform Lucene Index doesn't work 4 months 1 week ago #67449

  • GianlucaSedoc
  • GianlucaSedoc's Avatar
  • Offline
  • Elite Member
  • Posts: 173
  • Thank you received: 4
  • Karma: 0
Hi to all, actually i have a problem with Perform Lucene Index. The last run date without error is 28/02/2018, but in suitecrm.log shows nothing.
my version of suite is 7.9.7
How could i check errors or fix this problem?

Thanks
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 6 days ago #67512

  • pgr
  • pgr's Avatar
  • Offline
  • Administrator
  • Posts: 7776
  • Thank you received: 1208
  • Karma: 281
See this for plenty of ideas:

pgorod.github.io/Reindex-AOD/

Are your other scheduler jobs running successfully?

My SuiteCRM In-depth blog.
Thank you for always stating your SuiteCRM version, checking your logs, reading the Docs, and searching before you ask!
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 5 days ago #67572

  • GianlucaSedoc
  • GianlucaSedoc's Avatar
  • Offline
  • Elite Member
  • Posts: 173
  • Thank you received: 4
  • Karma: 0
pgr wrote:
See this for plenty of ideas:

pgorod.github.io/Reindex-AOD/

this is what i've done from the start
pgr wrote:

Are your other scheduler jobs running successfully?

yea they works, except those two. I've rebuilt the scheduler and:



shows me it worked on 08/06 but in last execution without error shows me "never"
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 4 days ago #67599

  • pgr
  • pgr's Avatar
  • Offline
  • Administrator
  • Posts: 7776
  • Thank you received: 1208
  • Karma: 281
So the job starts but trips on some error. You should have some message in php_errors.log...

It is likely related to ownerships/permissions on the Index directory. You need to look at the SetUID and SetGID bits on the parent directories, so that the newly created subdirectories get the right permissions also.

Maybe try this on the parent folder
find modules/AOD_Index/Index -type d -exec chmod 2775 {} \;

Then restart the indexing process.

By the way, what do you get when you type "umask" at the prompt?

My SuiteCRM In-depth blog.
Thank you for always stating your SuiteCRM version, checking your logs, reading the Docs, and searching before you ask!
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 4 days ago #67606

  • GianlucaSedoc
  • GianlucaSedoc's Avatar
  • Offline
  • Elite Member
  • Posts: 173
  • Thank you received: 4
  • Karma: 0
pgr wrote:
So the job starts but trips on some error. You should have some message in php_errors.log...

It is likely related to ownerships/permissions on the Index directory. You need to look at the SetUID and SetGID bits on the parent directories, so that the newly created subdirectories get the right permissions also.

Maybe try this on the parent folder
find modules/AOD_Index/Index -type d -exec chmod 2775 {} \;

Then restart the indexing process.


By the way, what do you get when you type "umask" at the prompt?

sadly in phph errors and in suitecrm.log no error about indexing or permission. I'm on windows server and the security permission are set up on IUSR with the max privilege. so i really don't know how to fix. Because if i edit a record, the index is created, but not with the script
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 4 days ago #67625

  • pgr
  • pgr's Avatar
  • Offline
  • Administrator
  • Posts: 7776
  • Thank you received: 1208
  • Karma: 281
Are your Scheduler jobs running under the same user that the Web server uses? It seems not...

My SuiteCRM In-depth blog.
Thank you for always stating your SuiteCRM version, checking your logs, reading the Docs, and searching before you ask!
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 3 days ago #67676

  • GianlucaSedoc
  • GianlucaSedoc's Avatar
  • Offline
  • Elite Member
  • Posts: 173
  • Thank you received: 4
  • Karma: 0
pgr wrote:
Are your Scheduler jobs running under the same user that the Web server uses? It seems not...





actually who start the schedulers is lybhrmcrm and he has all the permission
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 3 days ago #67680

  • pgr
  • pgr's Avatar
  • Offline
  • Administrator
  • Posts: 7776
  • Thank you received: 1208
  • Karma: 281
You need to get to the bottom of this. May I recommend SysInternals tools? You can download them for free from Microsoft website.

Use Process Explorer to really see which processes are executing under which user names.

Use Process Monitor to see the activity of your cron process and see if it's getting any access denied errors, and for which files or directories.

My SuiteCRM In-depth blog.
Thank you for always stating your SuiteCRM version, checking your logs, reading the Docs, and searching before you ask!
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 3 days ago #67697

  • GianlucaSedoc
  • GianlucaSedoc's Avatar
  • Offline
  • Elite Member
  • Posts: 173
  • Thank you received: 4
  • Karma: 0
pgr wrote:
You need to get to the bottom of this. May I recommend SysInternals tools? You can download them for free from Microsoft website.

Use Process Explorer to really see which processes are executing under which user names.

Use Process Monitor to see the activity of your cron process and see if it's getting any access denied errors, and for which files or directories.

the strange thing is that in the folder index there are file and everytime i edit a record from crm the index is stored correctly, i really don't get why it act like that
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 3 days ago #67746

  • GianlucaSedoc
  • GianlucaSedoc's Avatar
  • Offline
  • Elite Member
  • Posts: 173
  • Thank you received: 4
  • Karma: 0
pgr wrote:
You need to get to the bottom of this. May I recommend SysInternals tools? You can download them for free from Microsoft website.

Use Process Explorer to really see which processes are executing under which user names.

Use Process Monitor to see the activity of your cron process and see if it's getting any access denied errors, and for which files or directories.

A question. what is the function that goes to write on the table aod_indexevent? i think is that the problem. Nothing is writhing in this table
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 2 days ago #67790

  • pgr
  • pgr's Avatar
  • Offline
  • Administrator
  • Posts: 7776
  • Thank you received: 1208
  • Karma: 281
I don't know. I don't think you should be looking for the problem in the code, because we know this code works. You should be looking at configuration issues in your system.

My SuiteCRM In-depth blog.
Thank you for always stating your SuiteCRM version, checking your logs, reading the Docs, and searching before you ask!
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 1 day ago #67880

  • GianlucaSedoc
  • GianlucaSedoc's Avatar
  • Offline
  • Elite Member
  • Posts: 173
  • Thank you received: 4
  • Karma: 0
pgr wrote:
I don't know. I don't think you should be looking for the problem in the code, because we know this code works. You should be looking at configuration issues in your system.

Well actually i've fixed it by creating a scheduled task that launch ' /index.php?module=AOD_Index&action=optimise '
then i've made an entry point that goes in accounts, leads, and contacts table and create an aod_indexevent in the table for all the record missing.

Scheduled works so smooth and now the global search works perfect.
So at last i've fixed it, in some ways. and so, the problem isn't the permission, because if it was, the optimize doesen't start.

Btw thanks all. If needed, here my code:

db_connect.php
<?php

$serverName = "localhost\SQLEXPRESS2016";
$connectionInfo_CRMSuite = array( "Database"=>"*************",
    "UID"=>"sa",
    "PWD"=>"**********",
    "Encrypt"=>false,
    "TrustServerCertificate"=>false);
$connectionInfo_CRMSuite = sqlsrv_connect( $serverName, $connectionInfo_CRMSuite) or die ( print_r(sqlsrv_errors(), true) );
?>

fix_indexes.php
<?php
include "db_connect.php";



$errore = "inizio alle ore: ".date("H:i:s");
file_put_contents(__DIR__."\\log\\log_indexes\\logIndex.txt",$errore."\n",FILE_APPEND);
$count=0;

$query_account_id = "SELECT id,name FROM accounts where deleted=0 ";
$query_id = sqlsrv_query($connectionInfo_CRMSuite,$query_account_id);
while ($row_id = sqlsrv_fetch_array($query_id,SQLSRV_FETCH_ASSOC)){
    $name_value_list = [];
    $account_id = $row_id['id'];
    $checkQuerty= "SELECT id FROM aod_indexevent where record_id='$account_id'";
    $checkQuerty = sqlsrv_query($connectionInfo_CRMSuite,$checkQuerty);
    $checkQuerty = sqlsrv_fetch_array($checkQuerty,SQLSRV_FETCH_ASSOC);
    if(empty($checkQuerty)){
		try{
			$name_value_list['id'] = $account_id;
			$indexEvent = BeanFactory::newBean("AOD_IndexEvent");
			$indexEvent->record_id = $account_id;
			$indexEvent->name = iconv('','UTF-8',$row_id['name']);
			$indexEvent->record_module = 'Accounts';
			$indexEvent->update_date_modified = false;
			$indexEvent->date_modified = date("Y-m-d H:i:s");
			$indexEvent->success = true;
			$indexEvent->save();
		}catch(Exception $ex){
			$errore = $ex->getMessage();
			file_put_contents(__DIR__."\\log\\log_indexes\\logIndex_errore.txt",$errore."\n",FILE_APPEND);
        }
    }
}

$query_contacts_id = "SELECT id,first_name,last_name FROM contacts where deleted=0 ";
$query_id = sqlsrv_query($connectionInfo_CRMSuite,$query_contacts_id);
while ($row_id = sqlsrv_fetch_array($query_id,SQLSRV_FETCH_ASSOC)){
    $contact_id = $row_id['id'];
    $checkQuerty= "SELECT id FROM aod_indexevent where record_id='$contact_id'";
    $checkQuerty = sqlsrv_query($connectionInfo_CRMSuite,$checkQuerty);
    $checkQuerty = sqlsrv_fetch_array($checkQuerty,SQLSRV_FETCH_ASSOC);
    if(empty($checkQuerty)){
        try{
            $indexEvent = BeanFactory::newBean("AOD_IndexEvent");
            $indexEvent->record_id = $contact_id;
            $indexEvent->name = iconv('','UTF-8',$row_id['first_name'])." ".iconv('','UTF-8',$row_id['last_name']);
            $indexEvent->record_module = 'Contacts';
            $indexEvent->update_date_modified = false;
            $indexEvent->date_modified = date("Y-m-d H:i:s");
            $indexEvent->success = true;
            $indexEvent->save();
        }catch(Exception $ex){
            $errore = $ex->getMessage();
			file_put_contents(__DIR__."\\log\\log_indexes\\logIndex_errore.txt",$errore."\n",FILE_APPEND);
        }
    }
}



$query_contacts_id = "SELECT id,first_name,last_name FROM leads where deleted=0 ";
$query_id = sqlsrv_query($connectionInfo_CRMSuite,$query_contacts_id);
while ($row_id = sqlsrv_fetch_array($query_id,SQLSRV_FETCH_ASSOC)){
    $lead_id = $row_id['id'];
    $checkQuerty= "SELECT id FROM aod_indexevent where record_id='$lead_id'";
    $checkQuerty = sqlsrv_query($connectionInfo_CRMSuite,$checkQuerty);
    $checkQuerty = sqlsrv_fetch_array($checkQuerty,SQLSRV_FETCH_ASSOC);
    if(empty($checkQuerty)){
        try{
            $indexEvent = BeanFactory::newBean("AOD_IndexEvent");
            $indexEvent->record_id = $lead_id;
            $indexEvent->name = iconv('','UTF-8',$row_id['first_name'])." ".iconv('','UTF-8',$row_id['last_name']);
            $indexEvent->record_module = 'Contacts';
            $indexEvent->update_date_modified = false;
            $indexEvent->date_modified = date("Y-m-d H:i:s");
            $indexEvent->success = true;
            $indexEvent->save();
        }catch(Exception $ex){
            $errore = $ex->getMessage();
			file_put_contents(__DIR__."\\log\\log_indexes\\logIndex_errore.txt",$errore."\n",FILE_APPEND);
        }
    }
}


$errore = "Fine alle ore: ".date("H:i:s");
file_put_contents(__DIR__."\\log\\log_indexes\\logIndex.txt",$errore."\n",FILE_APPEND);
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 20 hours ago #67888

  • pgr
  • pgr's Avatar
  • Offline
  • Administrator
  • Posts: 7776
  • Thank you received: 1208
  • Karma: 281
Wow, that's an elaborate solution.

I'd like to get to the bottom of this... if you increase your log level to debug, and look for the queries writing into aod_indexevent, can you figure out what is wrong?

My SuiteCRM In-depth blog.
Thank you for always stating your SuiteCRM version, checking your logs, reading the Docs, and searching before you ask!
The administrator has disabled public write access.

Perform Lucene Index doesn't work 4 months 16 hours ago #67918

  • GianlucaSedoc
  • GianlucaSedoc's Avatar
  • Offline
  • Elite Member
  • Posts: 173
  • Thank you received: 4
  • Karma: 0
pgr wrote:
Wow, that's an elaborate solution.

I'd like to get to the bottom of this... if you increase your log level to debug, and look for the queries writing into aod_indexevent, can you figure out what is wrong?

Sadly i can't put the level of debug cause the site is used a lot, i've tried once, and in like 30 sec i got a suitecrm.log of 20mb. Is impossibile to manage this
The administrator has disabled public write access.
Time to create page: 0.073 seconds
Powered by Kunena Forum