Adobe Community: Message List - Solr

Re: Why Solr does not index PDF files that are in the same folder?

forums_noreply@adobe.com — Thu, 22 May 2014 17:52:48 GMT

Did you figure out why the pdfs would not index? I'm having the same problem, but only on my local machine.

KeywordTokenizerFactory splits the string for the exclamation mark

forums_noreply@adobe.com — Tue, 13 May 2014 18:27:58 GMT

Hi All

I have a following field settings in solr schema

<field name="<b>Exact_Word" omitPositions="true" termVectors="false" omitTermFreqAndPositions="true" compressed="true" type="string_ci" multiValued="false" indexed="true" stored="true" required="false" omitNorms="true"/>

As you can see Exact_Word has the KeywordTokenizerFactory and that should treat the string as it is.

Following is my responseHeader. As you can see I am searching my string only in the filed Exact_Word and expecting it to return the Word field and the score

"responseHeader":{

"status":0,

"QTime":14,

"params":{

"explainOther":"",

"fl":"Word,score",

"debugQuery":"on",

"indent":"on",

"start":"0",

"q":"d!sdasdsdwasd!asd@dsadsadas.edu",

"qf":"Exact_Word",

"wt":"json",

"fq":"",

"version":"2.2",

"rows":"10"}},

But when I enter email with the following string "d!sdasdsdwasdasd@dsadsadas.edu" it splits the string to two. I was under the impression that KeywordTokenizerFactory will treat the string as it is.

Following is the query debug result. There you can see it has split the word

"parsedquery":"+((DisjunctionMaxQuery((Exact_Word:d)) -DisjunctionMaxQuery((Exact_Word:sdasdsdwasdasd@dsadsadas.edu)))~1)",

can someone please tell why it produce the query result as this

If I put a string without the "!" sign as below, the produced query will be as below

"parsedquery":"+DisjunctionMaxQuery((Exact_Word:d_sdasdsdwasd_asd@dsadsadas.edu))",. This is what I expected solr to even with the "!" mark. with "_" mark it wont do a string split and treats the string as it is

I thought if the KeywordTokenizerFactory is applied then it should return the exact string as it is

Please help me to understand what is going wrong here

Why Solr does not index PDF files that are in the same folder?

forums_noreply@adobe.com — Fri, 25 Apr 2014 20:40:49 GMT

I have set up a Solr collection in CF 10 Administrator with following settings:

File Extensions: .*

Directory Path: C:/inetpub/wwwroot/test/

Recursively Index Sub Directories: unchecked

Return URL: http://localhost/test/

Language: Default ( I also tried "English" )

Inside the "C:/inetpub/wwwroot/test/" directory, I have 4 different types of files:.xls, .gif, .cfm, and .pdf. But every time when I try to update the index, none of the PDF fiels are indexed. All the other types of documents are indexed. I have looked at the PDF files' properties, none of them are "locked" or "protected".

I am using the Enterprise trial version with all the updates applied.

Anyone has experienced the same issue before?

Re: Solr Collection PDF display problem

forums_noreply@adobe.com — Mon, 07 Apr 2014 21:12:38 GMT

@Masterp, if you’ll view your response on the forums, you’ll see that unfortunately none of the code you posted was rendered (http://forums.adobe.com/message/6278444#6278444). Did you perhaps respond to the discussion via email? If so, I wonder if you were to go to the forum directly, and either editor your note or create a new one, and paste your code in there, maybe the web-based editor would turn your code into something that could be rendered properly for display.

/charlie

Re: Solr Collection PDF display problem

forums_noreply@adobe.com — Fri, 28 Mar 2014 18:59:58 GMT

Ah, ok. Your problem is the link you are showing to the user (file:///C:/inetpub/wwwroot/cfdocs/Newspaperfiles/Madison_County_Record/2013/05/18/Madiso n_County_Record_20130518.pdf).

That will not likely work in a browser. Not saying it can’t work, or will never work. But it’s not likely what you really should be using. Let me explain.

Look closely at it: a) it asks for a file:/// directive. Note that traditional browser URLs use http://. Now, it’s not that file:// is wrong. It’s that it’s not appropriate here. It would only work if the path pointed to existed on the user’s machinee. Do they have a folder called c:/inetpub/wwwroot? No. That’s on YOUR server. (And even if this was an intranet application, they would not see that file in that location, as it’s on the server. You’re not offering a path that would work from even other machines on the same network.)

I suspect what you need is to convert this URL into a “web accessible one”, and the good news (well, good and bad) is that the file you are naming is in fact in a web accessible directory, your inetpub\wwwroot. Assuming you can serve a test.cfm page from c:\inetpub\wwwroot using http://yourdomain/test.cfm, then you can serve these files as http://yourdomain/cfdocs/Newspaperfiles/Madison_County_Record/2013/05/18/Madison_County_Re cord_20130518.pdf.

Try that URL in your browser. If it works, then you just need to modify your code to provide THAT url to the user. To do that, you’d need to do some manipulation of the variable (you’re not showing us) which displays that link. If you’d share the line of code that outputs that link, we can provide code that will work. (Since you may be combining variables or functions to get the location you show now, I don’t want to just guess what might work.)

Hope that helps.

BTW, I said above that there was good news/bad news about this PDF being in a web accessible directory. It’s good for your need to just offer a link easily. It’s bad that if this is content that you may think you want to control in some way, such as to only show “some files” to “some people”, beware that anyone who sees one of these links (when we get it working) will be able to guess at other PDF file names. Just beware. (If you need to solve that, there are still other solutions, but they are way beyond the scope of this Solr forum.)

Indeed, as may be clear by now, your issue here (the problem you raised) isn’t really a Solr question at all. It’s more about fairly basic web and CF programming. Now, I don’t say that to embarrass you. Hey, we’ve all had a challenge like this some time in our coding lives. It’s just part of getting started with any particular challenge. My point though is that in asking for help here, in the Solr forum, you have limited your audience of help. Note that you’ve not gotten much (nor rapid) response. There are basic CFML forums and indeed several subforums for various aspects of working with CF. Just saying you may want to consider raising future questions there, if they really are of ongoing questions about basic application development. Hope that’s helpful.

/charlie

Re: Solr Collection PDF display problem

forums_noreply@adobe.com — Mon, 07 Apr 2014 17:20:57 GMT

I'm still not understanding below is my search code. Http://127.0.0.1

html>

Search Results

On Fri, Mar 28, 2014 at 12:00 PM, Charlie Arehart

Re: Solr Collection PDF display problem

forums_noreply@adobe.com — Fri, 28 Mar 2014 12:58:29 GMT

The ex. is below, but when I click on the link nothing happens Search

ResultsYour search returned 20 file(s).

File:

C:\inetpub\wwwroot\cfdocs\Newspaperfiles\Madison_County_Record\2013\05\18\Madison_County_R ecord_20130518.pdf<file:///C:/inetpub/wwwroot/cfdocs/Newspaperfiles/Madison_County_Record/ 2013/05/18/Madison_County_Record_20130518.pdf>

Document Title (if any): Madison County Record (May 18, 2013)

Score: 0.9018

summary: SATURDAY May 18, 2013 MADISON COUNTY MadisonCountyRecord.comRECORD

PUBLISHED EACH SATURDAY MADISON'S HOMETOWN NEWSPAPER SINCE 1967 25 CENTS

CLASSIFIEDS ONLINE POLL INSIDE Do you know a Madison resident graduating in

the next coming weeks? 20071207000856390 and other lands Section 19,

Township 3 South, Range 1 West, Huntsville Madison County, Alabama as same

is recorded in the office of the Judge of Probate of Madison County,

Alabama, in Document No.

On Thu, Mar 27, 2014 at 6:30 PM, Charlie Arehart

Re: Solr Collection PDF display problem

forums_noreply@adobe.com — Fri, 28 Mar 2014 01:30:47 GMT

Let’s start by you giving us an example of the URL value shown in the hyperlink you build. Maybe that will help explain why visiting it returns a blank page.

/charlie

Solr Collection PDF display problem

forums_noreply@adobe.com — Thu, 27 Mar 2014 20:30:32 GMT

I am having problems getting the Search collection I created to display the PDF files. When I perform a search I am able to get the pdf file listing put when I select a PDF file to display it it does nothing. Please help, below is the Search Action page I did

<head>

<title>Search Results</title>

</head>

<body>

<cfsearch

name= "newspaperarchive_results"

collection= "newspaperarchive"

criteria= "#Form.Criteria#">

ContextHighlightBegin="<b>"

ContextHighlightEnd="</b>"

ContextPassages="1"

ContextBytes="500"

maxrows= "100">

<h2>Search Results</h2>

Your search returned #newspaperarchive_results.RecordCount# file(s).

</cfoutput>

<p>

File: <a href="#url#">#key#</a><br>

Document Title (if any): #Title#<br>

Summary: #Summary#<b>

Highlighted Summary: #context#</p>

</cfoutput>

</body>

</html>

Problem with Coldfusion 9 and Solr

forums_noreply@adobe.com — Fri, 17 Jan 2014 21:11:29 GMT

Hi,

I have CF 9.0.1 installed, and I'm not able to define a Solr collection.
I get this error in the Administrator:
"An error occurred while creating the collection: org.apache.solr.common.SolrException. Check the Solr logs for more detail."
And in the Solr log, I get this:
"JNDI not configured for solr"

Any hint?

Thanks.

CFINDEX - Update existing collection with new data - duplicates records

forums_noreply@adobe.com — Mon, 16 Sep 2013 13:33:23 GMT

I have a collection made up of a document crawl to index pdf, word, excel etc.

now i need to update the existing collection with meta data related to articles in reference to the documents, however when i run the query to update the meta data the existing records do not get updated, a new record is created

initial index

<cfindex collection="#sCollectionName#"

action="update"

type="path"

key="\\192.168.168.66\media\#idx#\"

urlpath="#sFullSeverName#/wp-content/uploads/#idx#/"

extensions = "#lstValidFileExtensions#"

recurse="yes"

postterms_s = "" <--- used in later update --->

posttermssearch_s = ""<--- used in later update --->

posttermssearch_t = ""<--- used in later update --->

</cfif>

which creates about 1800 records

later afrer some manipulation nd cleanup (soem files need to be removed) i run

<cfindex

query="getTermsMembers"

collection="#sCollectionName#"

action="Update"

type="Custom"

key="KEY_LOOKUP"

body="url"

postterms_s = "terms"

posttermssearch_s = "terms_search"

posttermssearch_t = "terms_search"

i have confirmed that teh collection key and the value in KEY_LOOKUP are the same, but after running this block the number of records in teh collection doubles

CF10, Solr

1) is what i am trying to do possible?

2) what am i doing wrong?

Thanks

CF9 - Solr collection of database - Summaries are prepended with PK UID of record

forums_noreply@adobe.com — Tue, 22 Jan 2013 18:12:30 GMT

Hello, everyone.

I've got a CF9 server and created a few Solr collections from within CFAdmin, then used CFINDEX to update the collection with database records.

No matter what I do, when trying to access the Summary the data is prepended with the PK Unique ID of each record. Is there a way to prevent this behaviour?

Thank you,

^_^

Re: CFindex generating http 500 internal server error

forums_noreply@adobe.com — Tue, 22 Jan 2013 18:09:14 GMT

I would recommend removing ".cfm" from the list of files to index. We had that in one of our collections, and before we knew it searches were returning summaries that included SQL code, alerting everyone to things like database schema, etc.

^_^

Re: CFindex generating http 500 internal server error

forums_noreply@adobe.com — Mon, 14 Jan 2013 16:39:10 GMT

As another addendum, I tried indexing these same files on our new server which is running CF10, and it ran without a hitch so it seems to be a CF9 problem.

CFindex generating http 500 internal server error

forums_noreply@adobe.com — Mon, 14 Jan 2013 14:42:08 GMT

I'm currently on CF 9 and moving to 10 shortly so I've tried the "migrate" from Verity to SOLR. That created directories/files, but didn't move the data.

I then tried to index what I normally index to Verity, but with the slightly modified syntax. It runs briefly and gives a http 500 internal server error.

I tried just logging onto the server and running the index from the CF administrator. Same problem.

I tried re-creating the collection from scratch. No joy there either.

Some additional info. Looking at the log files I find:

"Error","jrpp-952","01/14/13","10:21:41","EventV3","org/apache/pdfbox/pdmodel/PDDocument The specific sequence of files included or processed is: D:\UTEWebsite\admin\indexwebsite.cfm, line: 15 "

I've tried removing the .pdf from the list but it didn't fix the problem. I have way too many files to rename all of the PDFs.

Weird SOLR error - missing required field_title

forums_noreply@adobe.com — Sat, 12 Jan 2013 01:40:31 GMT

I've got a FW/1 app deploying on CF9/Windows/Apache. Production is a cluster of 4 servers.

Code works fine on my local development machine, and when I push to our staging server.

When I push to production I get the following error when it trys to refresh my collection:

Original exception in onApplicationStart

The action public:main.default failed.

Document_61_missing_required_field_title Document_61_missing_required_field_title request: http://127.0.0.1:8983/solr/myproject/update?commit=true&waitFlush=false&waitSearcher=false&wt=javabin&version=1

(org.apache.solr.common.SolrException)

This code WAS working on production. This week a few of our servers were impacted by the CF security hack.

I have done the following:

Restarted SOLR
Deleted and recreated the collections in CFAdmin
Renamed the collection

Just curious if anyone has seen this error before? At this point I'm a bit stumped.

Jim

Shipped drivers and DIH

forums_noreply@adobe.com — Thu, 20 Dec 2012 09:14:59 GMT

We want to test DIH in Solr connecting to a MS SQL Server.

First we followed the steps described in the Help section "Indexing using Data import Handler" (http://help.adobe.com/en_US/ColdFusion/10.0/Developing/WSe61e35da8d318518-1acb57941353e8b4 f85-7ffd.html).

For connecting to MS SQL we would definitely prefer to use those drivers shipped with CF. As far as we could find our, for MS SQL this is the Macromedia Driver, so we switched the settings in data-config.xml to:

driver="macromedia.jdbc.MacromediaDriver" url="jdbc:macromedia:sqlserver://foo:1433;databaseName=bar"

We also added a lib/-subfolder into the folder of the corresponding collection where we did put the macromedia_drivers.jar so Solr could find the driver.

However, executing a fullimport fails.

SCHWERWIEGEND: Exception while processing: Mandator document : SolrInputDocument[{}]:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: select * from Mandators Processing Document # 1
    at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHand lerException.java:72)
    at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource .java:253)
    at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
    at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
    at org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:5 9)
    at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
    at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper. java:238)
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:596)
    at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:268)
    at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:187)
    at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359)
    at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427)
    at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
Caused by: macromedia.jdbc.MacromediaDriver$InvalidLicenseException: Der Gebrauch der Macromedia JDBC-Treiber ist eingeschrдnkt. Die Treiber dьrfen nur mit Macromedia-Serverprodukten verwendet werden. Prьfen Sie bitte, ob Ihre Lizenz die erforderlichen Treiber unterstьtzt.
    at macromedia.jdbc.MacromediaDriver.unlock(Unknown Source)
    at macromedia.jdbc.MacromediaDriver.connect(Unknown Source)
    at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:161)
    at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:128)
    at org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:363)
    at org.apache.solr.handler.dataimport.JdbcDataSource.access$200(JdbcDataSource.java:39)
    at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource .java:240)
    ... 11 more

Which indicates insufficient license priveliges.

We are running ColdFusion 10,282462 Developer Edition on Windows 7. We can successfully connect to the same database within our CF-apps and thus would have expected new DIH-actions of cfindex should be able to connect to the same databases with the same shipped drivers, too.

Anyone any thoughts how to fix this issue?

Maximum number of cores

forums_noreply@adobe.com — Wed, 05 Dec 2012 04:49:46 GMT

Hi,

I am using solr4.0, i have created 10 cores in solr. I want to know how many maximum number of cores can be created in solr.

migrating solr 3.6 to solr4.0

forums_noreply@adobe.com — Tue, 04 Dec 2012 06:28:24 GMT

Hi,

I had solr3.6 installed on my system, now i am migrating my solr3.6 to solr4.0. but i am getting the error

SEVERE: Unable to create core: collection1

java.io.IOException: Can't find resource 'solrconfig.xml' in classpath or 'solr/collection1/conf/', cwd=/opt/tomcat/bin

i don't know how to resolve this. Please help

Re: How to get maximum value of a field from an Indexed file in solr

forums_noreply@adobe.com — Mon, 26 Nov 2012 17:56:14 GMT

Are you needing the ID so you know what the next contentid should be when assigned?

Re: How do you use docBoost in CFIndex

forums_noreply@adobe.com — Mon, 26 Nov 2012 17:53:26 GMT

While you can hardcode a value in the docBoost field as listed above, pointing it to a column in a query doesn't work. It does not throw an error but nothing is added to the collection.

How to get maximum value of a field from an Indexed file in solr

forums_noreply@adobe.com — Sat, 24 Nov 2012 12:45:13 GMT

I have indexed an XML file in solr, which looks like:

this file has around 100000 documents. what is the way to get maximum value from the field value "contentid" parsing the whole file.

How do you use docBoost in CFIndex

forums_noreply@adobe.com — Tue, 20 Nov 2012 22:53:01 GMT

Hello,

I'm trying to use docBoost with CFIndex and whenever I use it, none of the documents get added to the index. It doesn't throw an error but nothing gets added or updated in the collection. Here is an example:

<cfindex

query="getFiles"

collection="#variables.documentCollection#"

urlpath="URLPath"

action="UPDATE"

key="keyPath"

category="categories"

title="Title"

docboost="3"

productID_i="SecurityProdID"

restrictContractor_b="restrictContractor"

release_s="version"

added_dt="addedDate"

updated_dt="last_revision"

altTitle_t="Title"

boosted_i="prefValue"

type="FILE"

status="searchStatus" />

However, if I leave out docBoost the documents index fine. Anyone?