Message-ID: <1069395936.2990.1485851528475.JavaMail.confluence@ip-10-127-227-164> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_2989_1240385439.1485851528475" ------=_Part_2989_1240385439.1485851528475 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html Solr Bundle

Solr Bundle

Solr Search Engine Bundle is bu= ndled by default in eZ Platform and these instructions are for configuring = it, for eZ Publish Platform 5.4 instructions see Solr Search Engine Bundle.

Version 1.0.x of Solr Bundle primarily aims to be used as replacem= ent for Legacy (SQL-based) search engine for better scalability and perform= ance (especially with field criteria and sort clauses). And while = it also provides better full text search thanks to Solr, more advance searc= h features like Faceting will come in later releases.

 

=20 =20

What is Solr Search Eng= ine Bundle?

ezplatform-solr-search-engine&n= bsp;as the package is called, aims to be a transparent drop in replacement = for the SQL based "Legacy" search engine powering Search API by default. By= enabling Solr and re-indexing your content, all your exising Search querie= s using SearchService, will be powered by Solr automatically. This allows y= ou to scale up your eZ Platform installation and be able to continue develo= pment locally against SQL engine, and have test infrastructure, Staging and= Prod powered by Solr. Thus remove considerable load from your database so = it can focus on more important things, like publishing 3D"(wink)".

Se Architecture page for further information on the architecture of eZ Platform.

How to set up Solr Search = engine

Step 1: Enabling the Bundle

Not needed with eZ Platform

This step is not needed for eZ Platform as of 15.09, however it is kept = here for reference in case you have previously disabled the bundle.

 

  1. Add/Update composer dependencies:

    command line
    =20
    composer require --no-update ezsystems/ezplatform-solr-search-engi=
    ne:~1.0
    composer update
    =20
  2. Activate EzPublishSolrSearchEngineBundle by adding the following lines = to your app/AppKernel.php file: new EzSys= tems\EzPlatformSolrSearchEngineBundle\EzSystemsEzPlatformSolrSearchEngineBu= ndle()

Step 2: Configurin= g & Starting Solr

 

Example here is for single core, look to=20 Solr=20 documentation for configuring Solr in other ways, also= see the provided configuration for some examples.

 

First, download and extract Solr, we currently support Solr 4.10= .4:

 

Secondly, copy configuration files needed for eZ Solr Search Engine bund= le, here from the root of your project to the place you extracted Solr<= /em>:

Command line example
=20
# Make sure to change the /opt/solr/ path with where you have plac=
ed Solr
cp -R vendor/ezsystems/ezplatform-solr-search-engine/lib/Resources/config/s=
olr/* /opt/solr/example/solr/collection1/conf/

/opt/solr/bin/solr start -f 
=20

 

Thirdly, Solr Bundle does not commit solr index changes direc= tly on repository updates, leaving it up to you to tune this using solrconfig.xml as best practice suggests, example config:<= /p>

solrconfig.xml
=20
<autoCommit>
  <!-- autoCommit is here left as-is like it is out of the box in Solr 4=
.10.4, this controls hard commits for durability/replication -->
  <maxTime>${solr.autoCommit.maxTime:15000}</maxTime>=20
  <openSearcher>false</openSearcher>=20
</autoCommit>

<autoSoftCommit>
  <!-- Soft commits controls mainly when changes becomes visible, by def=
ault we change value from -1 (disabled) to 100ms, to try to strike a balanc=
e between Solr performance and staleness of HttpCache generated by Solr que=
ries -->
  <maxTime>${solr.autoSoftCommit.maxTime:100}</maxTime>=20
</autoSoftCommit>
=20

 

Step 3: Configuring bundle

The Solr search engine bundle can be configured many ways, here are some= examples:

Single Core example (= default)

Out of the box in eZ Platform the following is enabled for simple setup:=

config.yml
=20
ez_search_engine_solr:
    endpoints:
        endpoint0:
            dsn: %solr_dsn%
            core: collection1
    connections:
        default:
            entry_endpoints:
                - endpoint0
            mapping:
                default: endpoint0
=20

Shared Core example

In the following example we have decided to separate one language as the= installation contains several similar languages, and one very different la= nguage that should be receive proper language analysis for proper stemming = and sorting behavior by Solr:

config.yml
=20
ez_search_engine_solr:
    endpoints:
        endpoint0:
            dsn: %solr_dsn%
            core: core0
       endpoint1:
            dsn: %solr_dsn%
            core: core1
    connections:
        default:
            entry_endpoints:
                - endpoint0
                - endpoint1
            mapping:
                translations:
                    - jpn-JP: endpoint1
                # Other languages, for instance eng-US and other western la=
nguages are sharing core
                default: endpoint0
=20

Multi Core example

If full language analysis features are preferred, then each language can= be configured to separate cores.

Note: Please make sure to test this setup against single core as it = might perform worse then single core if your project uses a lot for languag= e fallbacks per siteaccess as queries will then be perf= ormed across several cores at once.

config.yml
=20
ez_search_engine_solr:
    endpoints:
        endpoint0:
            dsn: %solr_dsn%
            core: core0
       endpoint1:
            dsn: %solr_dsn%
            core: core1
       endpoint2:
            dsn: %solr_dsn%
            core: core2
       endpoint3:
            dsn: %solr_dsn%
            core: core3
       endpoint4:
            dsn: %solr_dsn%
            core: core4
       endpoint5:
            dsn: %solr_dsn%
            core: core5
       endpoint6:
            dsn: %solr_dsn%
            core: core6
    connections:
        default:
            entry_endpoints:
                - endpoint0
                - endpoint1
                - endpoint2
                - endpoint3
                - endpoint4
                - endpoint5
                - endpoint6
            mapping:
                translations:
                    - jpn-JP: endpoint1
                    - eng-US: endpoint2
                    - fre-FR: endpoint3
                    - ger-DE: endpoint4
                    - esp-ES: endpoint5
                # Not really used, but specified here for fallback if more =
languages are suddenly added by content admins
                default: endpoint0
                # Also use separate core for main languages (differs from c=
ontent object to content object)
                # This is useful to reduce number of cores queried for alwa=
ys available language fallbacks
                main_translations: endpoint6
=20

 

Step 4: Configuring repository with the specific search engine

The following is an example of configuring Solr Search Engine, whe= re connection name is same as in example above, and = engine is set to solr:

 

ezplatform.yml
=20
ezpublish:
    repositories:
        main:
            storage:
                engine: legacy
                connection: default # This should be the connection you had=
 from before for repository
            search:
                engine: solr # One of legacy (default) or solr
                connection: default # If legacy same as storage applies, if=
 solr use same as you defined in step 3
=20

 

Step 5: Run CLI indexing = command

Make sure to configure your setup for indexing

Some exceptions might happen on indexing if you have not configured your= setup correctly, here are the most common issues you may encounter:

  • Exception if Binary files in database have an invalid path prefix
    • Make sure ezplatform.yml configuration  var_dir is configured properly. <= /li>
    • If your database is inconsistent in regards to file paths,= try to update entries to be correct (but make sure to make a backup fi= rst).
  • Exception on unsupported Field Types
    • Make sure to implement all Field Types in your installation, or t= o configure missing ones as NullType if implementation is not needed.
  • Content not immediately available 
    • Solr Bundle is on purpose not committing changes directly on Repo= sitory updates (on indexing), but letting you control this using S= olr configuration. Adjust Solr autoSoftCom= mit  visibility of change to search index) and= /or autoCommit (hard commit, for durability = and replication) to balance performance and load on your Solr ins= tance against needs you have for "NRT".
  • Running out of memory during indexing
    • In general make sure to run indexing using prod environment to av= oid debuggers and loggers from filing up memory.
    • Stash: Disable in_memory cache as recommended on Persistence cache for long running scripts.
    • Flysystem: An open issue exists where you can find further info https://jira.ez.no/browse/EZP-25325<= /a>

Last step is to execute initial indexation of data:

=20
php app/console --env=3Dprod --siteaccess=3D<name> ezplatfor=
m:solr_create_index
=20

Providing feedback

After completing the installation you are now free to use your site as u= sual. If you get any exceptions for missing features, have feedback on perf= ormance, or want to discuss, join our community slack channel at https://ezcommunity.slack.com/messages/ezplatfo= rm-use/

 

------=_Part_2989_1240385439.1485851528475 Content-Type: image/png Content-Transfer-Encoding: base64 Content-Location: file:///C:/450b34994881dc04bd093e73c25eda70 iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAA3NCSVQICAjb4U/gAAAAeFBMVEX/ ///ht0zds0vbsUrZr0rZr0rZr0rXrUr/01H90VH7z1D5zVD2zVP3y1D1yU/xx07tx1XwxU7vw07k wFfjuUzht0zhtkvftUvds0vbsUrQslvZr0rXrUrJrl3Vq0nAqF+3oWGllWScj2aUiWiKgmqBfGx4 dm5wcHAZd7u/AAAAKHRSTlMAEXe7u8zd7v////////////////////////////////////////// apo9sAAAAAlwSFlzAAALEgAACxIB0t1+/AAAABx0RVh0U29mdHdhcmUAQWRvYmUgRmlyZXdvcmtz IENTNui8sowAAAAUdEVYdENyZWF0aW9uIFRpbWUANi8xLzEzOKlF0AAAAJ1JREFUGJVFT1sCgjAM KyClbvIQ0Q0pU1Tk/je02wDztaRL2gAI0owFhxQiEhy6Wuu6GzAJvDCKArQtvIK3I61QBsXP69xa +cMp5J2Q+5toWeRhcnAVTYsnQTg7cFuCt5DivxBjHXBFn9MuiEVCx/nREF2eX6JrLms1NdMsua+R KlkLuB0qCRbD6X0ZedkXsQyyaUvdGo7l9vpZqP8DgbkMiplsfQgAAAAASUVORK5CYII= ------=_Part_2989_1240385439.1485851528475--