Incorrect prices appearing randomly in invoices [SOLVED]

Greetings,

We have encountered a potentially serious problem in 1.9 that we have not seen in previous versions.  This has now been caught twice: once by us after incorrectly charging and taking payment from a client, and the second by the client at the desk.  It may have occurred other times without being caught.

It seems that items are being assigned random pricing during invoice creation.  Checking the pricing data of individual items shows that all is correct, and so the error appears to occur during construction of the invoice itself.  It is not a document template issue as the wrong price and total (as shown on the invoice) is appearing in the client's account.

I have attached copies of the two incorrect (and corrected) invoices. 

I have asked if anything unusual occurred during creation of the invoices but there does not appear to be any unusual editing or any other action that would have caused the problem.  So unfortunately, at this stage, the error is not reproducible.  The only characteristic of note is that in both of the invoices (small sample!), it was only the first item on the invoice that attracted the seemingly random prices.

I'm sorry that I cannot provide any further hints as to the cause at this stage. There are no adjacent items (either alphabetically or by ID number) that share the erroneous prices, and the bogus prices do not seem to have any systematic relationship to the correct ones. 

We will have a staff meeting and direct everyone to be vigilant, but I suspect that small errors (such as in the blood collection fee) will likely go unnoticed.  Only the larger errors ($156.70 for 5 Apo-Doxy tabs for example) will likely be caught.

I'm nervous.

Thanks,

Sam

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Re: Incorrect prices appearing randomly in invoices

Sam - your pdf contains 4 invoices, 103699, 103728, 108242 & 108242 - although the first two call out the same stuff (and have the same date) they are different invoices - whereas the last two are the same invoice.

What price shows on the screen for 108242?

It should be possible to write some SQL to run through the system looking at the line item amounts in the invoices and comparing these with the actual product prices.  This might show up a pattern to help understand what is happening. I will see what I can generate.

Regards, Tim G

 

Re: Incorrect prices appearing randomly in invoices

Tim...

103699 was the one where the error was detected only after the client had left and after paying the invoice.  So that one had been finalized and paid.  When the error was detected, that invoice was reversed and a new one issued as 103728. 

108242 was the one where the error was detected by the client after it had been printed, but before being finalized. A copy of that first 108242, marked Error, is attached. In that case, because it had not yet been finalized, the receptionist edited the invoice, altering the line item by hand.  That is attached as the second 108242, and is the one that currently exists in the system.  She didn't quite get the pricing correct when entering by hand, but it is close. My suggestion at the time would have been to finalize 108242, reverse and re-issue as above, but I was not there when all this was happening.  In any case, the problem is with the production of 103699 and of the original 108242, not with her (well, almost correct ) corrections.  Testing shows that creating a new, identical invoice with the same line items as the erroneous invoices, produces a correct invoices as expected.  I can't reproduce the errors.

I have attached screenshots of the pricing pages for the Blood Collection Fee and Apo-Doxy.  The blood collection fee appears to have remained unchanged since entering it into OVPMS shortly after opening the practice in January 2015.  The Apo-Doxy unit price also appears to have remained unchanged from the original, although I believe one of the vets went through and adjusted dispensing fees in early 2016. I suspect he did so by editing the prices only, without ending one pricing scheme and starting another, as would probably have been better advised.

An SQL query comparing invoices to prices would probably be most useful as it might not only help track down the cause of the problem, but reveal to us how widespread the problem actually is.  Nothing like this ever caught our eye prior to 1.9 so gratefully, I am concerned only for invoices created over the past few months.  If you remember, I did have migration problems with the 1.8 database, resolved as a case-sensitivity issue in the migration script.  I did, however, followed your cautions at the time and did a totally fresh migration from 1.8 with your updated and re-posted script after that issue was solved.

I obviously do not know about the innards of how the invoicing routine works, but from just a logic  perspective, it seems as if a value is on occasion, not being cleared when a new invoice is being produced.  It just seems to be collecting garbage in that first (printed at least) item (?).

Thanks for taking a look...

Sam

AttachmentSize
BloodCollectionPrice.jpg 57.13 KB
Apo-DoxyPrice1.jpg 122.95 KB

Re: Incorrect prices appearing randomly in invoices

Do you have multiple fixed or unit prices for any of the line items that have errors?

For unit prices, check that the date ranges are correct.

Re: Incorrect prices appearing randomly in invoices

Can you send the output of the following to tanderson at openvpms.org ?

select invoice.act_id invoice_id,
    item.act_id item_id,
    item.activity_start_time,
    product.entity_Id product_id,
    product.name,
    fitem.fixed_amount,
    fitem.quantity,
    fitem.unit_amount,
    fitem.total total,
    author.entity_id author_id,
    author.name author_name
from acts invoice
join act_relationships r
    on invoice.act_id = r.source_id
join acts item
    on r.target_id = item.act_id
join participations p
    on p.act_id = item.act_id
        and p.arch_short_name = 'participation.product'
join entities product
    on p.entity_id = product.entity_id
left join participations pa
    on pa.act_id = item.act_id
        and pa.arch_short_name = 'participation.author'
left join entities author
    on pa.entity_id = author.entity_id
join financial_acts fitem on item.act_id = fitem.financial_act_id
where invoice.arch_short_name = 'act.customerAccountChargesInvoice'
    and invoice.act_id in (103699, 103728, 108242)
order by invoice.act_id, item.activity_start_time, item.act_id;

Also:

  • are lines with incorrect pricing being produced by a template or manual entry?
  • do you allow modification of fixed or unit prices on invoices?

Re: Incorrect prices appearing randomly in invoices

Tim...

I've sent off the results of the MySQL query above as requested.

1) The lines with the incorrect pricing were produced from invoices that were manually entered as we have not yet started using item templates.

2) We do allow modification of the prices, and in fact that's how the fix was made on invoice 108242... the erroneous price was adjusted back to the correct (almost) value by hand. 

3) We do not use any linked pricing.

Sam

Re: Incorrect prices appearing randomly in invoices

Sam/Tim A - I have most of the SQL needed to check the invoice prices against the product's. It found lots of mismatches - but on investigation a lot are due to the use of linked prices. Will enhance the SQL to allow for these.

Regards, Tim G

Re: Incorrect prices appearing randomly in invoices

OK - here is the SQL. I hope that someone can see an error in the logic, because it is finding a lot of errors - though the one I checked looks real.  That is, I have a horrible feeling that we have a real problem.

select
    p.name as prodName, a.act_id as invoiceID, date(a.activity_start_time) as invDate,  a.status,
    cu.entity_id as custID, fa.quantity as qty,
    fa.fixed_amount as fPriceInvoice, fp.price as fPriceProduct, fp.name as fpName, fp.type as fpType,
    fa.unit_amount as uPriceInvoice,
    up.price as uPriceProduct,
    concat ((if (up.price <> fa.unit_amount,"UP??",""))," ",if (((fa.fixed_amount > 0) and isnull(fp.price)),"FP??","")) as error
from acts a
join act_relationships ar on a.act_id = ar.source_id and ar.arch_short_name = 'actRelationship.customerAccountInvoiceItem'
join acts ai on ai.act_id = ar.target_id
join financial_acts fa on fa.financial_act_id = ai.act_id
join participations pp on pp.act_id = ai.act_id and pp.arch_short_name = 'participation.product'
join entities p on p.entity_id = pp.entity_id
join product_prices up on up.product_id = p.entity_id and up.arch_short_name ='productPrice.unitPrice'
    and ((up.start_time is null or up.start_time <= a.activity_start_time) and (up.end_time is null or up.end_time > a.activity_start_time))
left join (
    select fp.price as price, p.entity_id as pid, fp.start_time, fp.end_time, fp.name, 'fp' as type
        from entities p
        join product_prices fp on fp.product_id = p.entity_id and fp.arch_short_name ='productPrice.fixedPrice'
        where p.arch_short_name in ('product.merchandise','product.medication','product.service')
    union
     select tp.price as price, p.entity_id as pid, tp.start_time, tp.end_time, tp.name, 'tp' as type
        from entities p
        join entity_relationships erl on erl.source_id = p.entity_id and erl.arch_short_name = 'entityRelationship.productLink'
        join entities pt on pt.entity_id = erl.target_id
        join product_prices tp on tp.product_id = pt.entity_id and tp.arch_short_name ='productPrice.fixedPrice'
        where p.arch_short_name in ('product.merchandise','product.medication','product.service')
    )as fp    on fp.pid = p.entity_id and fp.price = fa.fixed_amount
        and ((fp.start_time is null or fp.start_time <= a.activity_start_time) and (fp.end_time is null or fp.end_time > a.activity_start_time))
join participations pc on pc.act_id = a.act_id and pc.arch_short_name = 'participation.customer'
join entities cu on cu.entity_id = pc.entity_id
where a.arch_short_name = 'act.customerAccountChargesInvoice'
and a.activity_start_time > '2017-02-01'
and a.act_id = 4145211
and p.entity_id = 47216
and (up.price <> fa.unit_amount or (fa.fixed_amount > 0 and isnull(fp.price)))
order by prodName,invDate;

Note the extra bits of selection at the bottom to trim down to the data presented here, ie the date must be after 1Feb2017, I am looking at one invoice, and one product.

The logic of the above is:

- for the unit prices, just compare the invoice and product price

- for the fixed prices, generate a table of all fixed and Price Template prices, and left join to that matching on product, price and date - a null result means that the fixed price on the invoice was not in the set of allowed fixed prices for the product.

This finds:

So we have two errors in each of these line items - the unit price does not match (31 vs 26) and the fixed price on the invoice ($60) does not occur is the set for the product - which using the following SQL

select fp.price as price, p.entity_id as pid, fp.start_time, fp.end_time, fp.name, 'fp' as type
    from entities p
    join product_prices fp on fp.product_id = p.entity_id and fp.arch_short_name ='productPrice.fixedPrice'
    where p.arch_short_name in ('product.merchandise','product.medication','product.service')
    and p.entity_id = 47216
union
 select tp.price as price, p.entity_id as pid, tp.start_time, tp.end_time, tp.name, 'tp' as type
    from entities p
    join entity_relationships erl on erl.source_id = p.entity_id and erl.arch_short_name = 'entityRelationship.productLink'
    join entities pt on pt.entity_id = erl.target_id
    join product_prices tp on tp.product_id = pt.entity_id and tp.arch_short_name ='productPrice.fixedPrice'
    where p.arch_short_name in ('product.merchandise','product.medication','product.service')
        and p.entity_id = 47216;

are:

Here is the product:

so just a unit price.  Here is the linked item:

Here is the invoice:

So - I have a horrible feeling that Sam is correct - the system is grabbing the wrong prices somehow.

I also ran a query to print the pricing for the Acp 2 Injection since 1/1/2015 - it appears in 84 invoice line items and some 46 of these appear to be wrong - ie are not $50 fixed, $26 unit. The csv file (renamed to .txt) is attached.

If this analysis is correct, then it is not a 'new in 1.9' problem.

Regards, Tim G

AttachmentSize
priceMismatch.txt 8.42 KB

Re: Incorrect prices appearing randomly in invoices

OK - here is the SQL. I hope that someone can see an error in the logic, because it is finding a lot of errors - though the one I checked looks real.  That is, I have a horrible feeling that we have a real problem.

select
    p.name as prodName, a.act_id as invoiceID, date(a.activity_start_time) as invDate,  a.status,
    cu.entity_id as custID, fa.quantity as qty,
    fa.fixed_amount as fPriceInvoice, fp.price as fPriceProduct, fp.name as fpName, fp.type as fpType,
    fa.unit_amount as uPriceInvoice,
    up.price as uPriceProduct,
    concat ((if (up.price <> fa.unit_amount,"UP??",""))," ",if (((fa.fixed_amount > 0) and isnull(fp.price)),"FP??","")) as error
from acts a
join act_relationships ar on a.act_id = ar.source_id and ar.arch_short_name = 'actRelationship.customerAccountInvoiceItem'
join acts ai on ai.act_id = ar.target_id
join financial_acts fa on fa.financial_act_id = ai.act_id
join participations pp on pp.act_id = ai.act_id and pp.arch_short_name = 'participation.product'
join entities p on p.entity_id = pp.entity_id
join product_prices up on up.product_id = p.entity_id and up.arch_short_name ='productPrice.unitPrice'
    and ((up.start_time is null or up.start_time <= a.activity_start_time) and (up.end_time is null or up.end_time > a.activity_start_time))
left join (
    select fp.price as price, p.entity_id as pid, fp.start_time, fp.end_time, fp.name, 'fp' as type
        from entities p
        join product_prices fp on fp.product_id = p.entity_id and fp.arch_short_name ='productPrice.fixedPrice'
        where p.arch_short_name in ('product.merchandise','product.medication','product.service')
    union
     select tp.price as price, p.entity_id as pid, tp.start_time, tp.end_time, tp.name, 'tp' as type
        from entities p
        join entity_relationships erl on erl.source_id = p.entity_id and erl.arch_short_name = 'entityRelationship.productLink'
        join entities pt on pt.entity_id = erl.target_id
        join product_prices tp on tp.product_id = pt.entity_id and tp.arch_short_name ='productPrice.fixedPrice'
        where p.arch_short_name in ('product.merchandise','product.medication','product.service')
    )as fp    on fp.pid = p.entity_id and fp.price = fa.fixed_amount
        and ((fp.start_time is null or fp.start_time <= a.activity_start_time) and (fp.end_time is null or fp.end_time > a.activity_start_time))
join participations pc on pc.act_id = a.act_id and pc.arch_short_name = 'participation.customer'
join entities cu on cu.entity_id = pc.entity_id
where a.arch_short_name = 'act.customerAccountChargesInvoice'
and a.activity_start_time > '2017-02-01'
and a.act_id = 4145211
and p.entity_id = 47216
and (up.price <> fa.unit_amount or (fa.fixed_amount > 0 and isnull(fp.price)))
order by prodName,invDate;

Note the extra bits of selection at the bottom to trim down to the data presented here, ie the date must be after 1Feb2017, I am looking at one invoice, and one product.

The logic of the above is:

- for the unit prices, just compare the invoice and product price

- for the fixed prices, generate a table of all fixed and Price Template prices, and left join to that matching on product, price and date - a null result means that the fixed price on the invoice was not in the set of allowed fixed prices for the product.

This finds:

So we have two errors in each of these line items - the unit price does not match (31 vs 26) and the fixed price on the invoice ($60) does not occur is the set for the product - which using the following SQL

select fp.price as price, p.entity_id as pid, fp.start_time, fp.end_time, fp.name, 'fp' as type
    from entities p
    join product_prices fp on fp.product_id = p.entity_id and fp.arch_short_name ='productPrice.fixedPrice'
    where p.arch_short_name in ('product.merchandise','product.medication','product.service')
    and p.entity_id = 47216
union
 select tp.price as price, p.entity_id as pid, tp.start_time, tp.end_time, tp.name, 'tp' as type
    from entities p
    join entity_relationships erl on erl.source_id = p.entity_id and erl.arch_short_name = 'entityRelationship.productLink'
    join entities pt on pt.entity_id = erl.target_id
    join product_prices tp on tp.product_id = pt.entity_id and tp.arch_short_name ='productPrice.fixedPrice'
    where p.arch_short_name in ('product.merchandise','product.medication','product.service')
        and p.entity_id = 47216;

are:

Here is the product:

so just a unit price.  Here is the linked item:

Here is the invoice:

So - I have a horrible feeling that Sam is correct - the system is grabbing the wrong prices somehow.

I also ran a query to print the pricing for the Acp 2 Injection since 1/1/2015 - it appears in 84 invoice line items and some 46 of these appear to be wrong - ie are not $50 fixed, $26 unit. The csv file (renamed to .txt) is attached.

If this analysis is correct, then it is not a 'new in 1.9' problem.

Regards, Tim G

Re: Incorrect prices appearing randomly in invoices

Your script doesn't appear to take into account:

  • cases where the quantities <> 1

The product unit price needs to be multiplied by the quantity before comparing it to that on the item.

  • zero price items - i.e. where the produce has been included by a template with Zero Price set
  • service ratios - the service ratio is applied before calculating the tax inclusive price
  • tax - the tax inclusive price is stored on the item, whereas the tax exclusive price is stored on the product

 

Re: Incorrect prices appearing randomly in invoices

Tim - I agree that it does not handle service ratios or zero priced templated items but neither of these are applicable here.

Quantity: as far as I can see, the financial_acts record unit_amount field holds the unit price, not the unit price multiplied by the quantity.  [But the screen does show the latter]

Tax: again, the unit_amount in the financial_acts record is ex-tax - but the screen shows the inc-tax amount. Hence if we have a product with an ex-tax unit price of $1 and a tax rate of 10%, and we sell 20 of them then the screen will show the unit component as $1 x 1.10 x 20 = $22 but the financial_acts record will have unit_amount = $1

I still think that we have a problem.

Regards, Tim G

Re: Incorrect prices appearing randomly in invoices

Tim - I agree that it does not handle service ratios or zero priced templated items. I thought that we didn't use service ratios - but we do - the overnight emergency service makes heavy use of them.  Also we use the minimum price facility (round to nearest 50c) and this needs to factored in if there is a service ratio. I will see if I can add these into the SQL code.

Quantity: as far as I can see, the financial_acts record unit_amount field holds the unit price, not the unit price multiplied by the quantity.  [But the screen does show the latter]

Tax: agreed (zero tax in Hong Kong so is not a problem in my system)

Will report back when I have tweaked the SQL.

Regards, Tim G

Re: Incorrect prices appearing randomly in invoices

Quantity: you are correct. I had a brain fade.

-Tim A

Re: Incorrect prices appearing randomly in invoices

I haven't been able to reproduce this locally, and it doesn't appear to be an issue in one of the large production databases I have access to.

If it is occurring, I would expect it to be related to one of the following circumstances:

  • multiple unit prices on a product, with the same date range
  • configuration of service ratios associated with the product's type
  • configuration of pricing groups associated with the products fixed or unit prices
  • an error being generated when changing from one product to another that somehow prevents the price update. Note that I can't trigger this behaviour, but if it is happening, there should be an error in the openvpms-full.log.

Except in the case of service ratios, I would expect there to be a product price in the system that corresponds to the incorrect prices you are seeing.

Re: Incorrect prices appearing randomly in invoices

Tim...

Thanks for the follow-up on this.  Just wanted to explore the possible causes you have noted:

1) For the products we have encountered with the bogus pricing... there are no multiple unit prices

2) We have not configured or use service ratios

3) We do not have or use any pricing groups, and do not use any linked prices or product templates

4) I have examined the log for Feb 10 (attached), the day where 5 Apo-Doxy tabs were invoiced as $164.54 tax-inc, but see nothing sinister in it.  What kind of error message would one expect to appear... as I can write a script to scan them regularly for that error.

I know Tim G. was working at one point on an SQL query that compared pricing wth the actual invoice.  While many of the discrepancies he found could be explained away, I think having such a validator available would be a great addition to the toolbox.

Thanks... we'll keep being vigilant.

Sam   

 

 

AttachmentSize
Feb10log.txt 27.76 KB

Re: Incorrect prices appearing randomly in invoices

I got to thinking that may not be a good test as that Apo-Doxy invoice was the one we caught before being finalized.  The better test is the one with the Blood Collection Fee (Inv 103699) where it was finalized with the wrong price.  I've extracted the log for that day and have attached it.

Thanks,

Sam

AttachmentSize
19Jan2017-log.txt 37.88 KB

Re: Incorrect prices appearing randomly in invoices

I can't see anything untoward in that log.

-Tim

Re: Incorrect prices appearing randomly in invoices

An 'Invoice Price Check' report has been developed (see attached) which enables this situation to be investigated. Running this over the Hong Kong system has found no evidence of underlying systematic errors in the OpenVPMS code.

Where "errors" have been detected, investigation shows that these are due to two things:

  1. staff changing the fixed and unit amounts on the invoice line item as it is entered/edited. [For example, where they want to charge a lower amount than the standard, rather than entering a discount amount, they simply over-write the fixed or unit amount with a lower value. We also found cases where the amounts had been increased because the staff knew that that the product price was wrong (was set to $80 - should have been $800).
  2. Price changes being done without 'closing off' the old price and entering a new one with new applicability dates.

The attached report (with its US Letter version) will probably have more work done to it and at some point in time I will add it to the resource library.  However, it is useful for keeping an eye whether any pricing irregularities are occuring. Note that you will probably was to set the 'ignore differences under' numbers to somethuing larger than the $1.00 default - I have been using $200 (about $AU 34) to check the Hong Kong system - otherwise it will detect errors due to incorrectly done price changes.

Note that the report is for 1.9 systems - will not work correctly on 1.8 and earlier.

Regards, Tim G

AttachmentSize
RPT Invoice Price Check.jrxml 35.85 KB
RPT Invoice Price Check-US.jrxml 35.71 KB

Re: Incorrect prices appearing randomly in invoices

Tim,

Thanks very much for this report.  I have run it over 2015 and detected no errors, the first errors only appearing starting January 2016 after one of our vets made price edits without closing out old prices... only editing old ones.  A common edit was to eliminate/add fixed prices for items for which dispensing fees were to be removed or added.  Some crazy edits may also have occurred.

I am now confident that there are no programmatic / systematic problems, but am feeling that there is perhaps more of a sensitivity to not closing out old prices than suspected.  My feeling is now that one must absolutely close out old prices before entering new ones... a practice we have now adopted universally at our clinic.

With your report, we should be able to identify those items that are problematic.  The question now remains what to do about it?  I would think that inactivating those products and recreating them would perhaps be the way to go.  Any thoughts on this?

Thank you so much for all your dedicated work on this. We really had our confidence shaken, but now see that all of our problems were self-created.  I guess that everyone should RTFM - Read The Fine Manual.

Sam

 

Re: Incorrect prices appearing randomly in invoices

Greetings,

I have run a slightly modified Invoice Price Check Report and scanning through its 113 pages, estimate that approximately 500+ items are being snagged.  While not all of them are problematic, it looks like a substantial number of them (mostly medications) will need attention. 

I'm looking for advice on how to proceed. 

Three possibilities come to mind:

1) For all problematic items, close out all existing prices and open new fixed and unit prices with today's starting date;

2) Inactivate all problematic items and completely re-enter each item by hand;

3) Use the Copy button to copy all tab information from a problematic item to a new one, inactivate the old one and rename the (Copy).

Technique (1) would be relatively easy, but from interviews, it appears that some of our crazier invoicing errors may stem from clinicians and staff 're-purposing' items.  In other words, when doing updates, they have simply changed the name on an existing item from one we don't need anymore, to a new item. (Groan).  It may not be possible to identify all of the items that have been so treated. 

(2) Re-entering by hand would be a gargantuan (costly) project prone to error.  Many of the items are medications with detailed dispensing notes. 

(3) Using the Copy button would of course be the easiest and most accurate way to go.  My question, however, has to do with what actually is being copied.  If all of the pricing problems with an item are just being regenerated under a new ID number, then obviously I'm not getting anywhere with a copy.  I've noticed, however, that when reviewing the prices of troublesome items on the screen, all seems good.  So if copying only duplicates those data that are displayed, then copying may be the way to go.  I suspect that this may be the case, but am reluctant to proceed without confirmation.

If it is decided that using the Copy function is advisable, then what would be your guidance on fixed and unit pricing in the newly-copied item? We don't anywhere have the situation where overlapping prices exist on the screen.  In this case, would it be OK to leave the prices alone, or should all copied prices be closed out and new ones entered as in 1) above?

I would appreciate any advice on this.  My feeling is that the answer to effective error-correction requires a deeper understanding of what is going on behind the scenes than what I have.

Thanks,

Sam

 

Re: Incorrect prices appearing randomly in invoices

Sam - I have just sent you the latest (and I hope final) version of the price check program.  As I said in my email, I am still looking at some funnies in our system, but I am not yet convinced that there are any systemematic problems.

Your story of 'repurposing' items seems really scary - looking at Fido's old records will now show that he was treated with the 'new' item, whereas he was actually treated with the old one.

I personally would not do anything about resetting prices in the database.

I would however:

  1. remove the ability to change prices in the system from all who do not understand EXACTLY how this should be done
  2. run the Invoice Price Check report every day (or couple of days) checking the invoices issued in days since the last run. This should allow you to check & investigate things which people still remember doing them.

Do you want me to build a report to find products where there are multiple unclosed off prices?  Given that the documentation says that in these cases the system uses the oldest price, it would be worth cleaning these up.

Regards, Tim G

Re: Incorrect prices appearing randomly in invoices

Hi Tim,

Thanks for getting back. 

1) I believe that I have put the "Fear of Sam" into everyone enough that there should be no one adjusting any prices (except me) for the foreseeable future.  In our case, I don't think I will need to formalize it. I am sure that my challenge will be in ever getting them to change another price again!

2) I will definitely be running the report on a regular basis.  Just this week, we caught a $15 fluorescein stain going out the door for $565.  That again got caught at the pre-finalization stage but smaller errors are probably not.  

But the report is not a fix.  When you say "I personally would not do anything about resetting prices in the database", I hope you mean ... "at the present time." I can't see that we would be able to proceed like this knowing that there is a serious fault in our DB that is causing errors in the invoice/record.  I will continue to track the errors through the report, but I feel that the value of the report is to indicate which items and/or prices need to be purged, either through item inactivation or through closing/opening prices.  I am less concerned about the historical accuracy... but feel that cleaning this up moving forward is imperative.

I do not believe that we have any "visible-on-the-screen" overlapping, unclosed-off prices in our pricing catalog.  But if you mean prices that were changed without the close/open procedure being applied, leaving invisible multiple prices in the DB, than a new report to catch those would be fantastic.  I guess that's ultimately what I am trying to find through the present report... problematic items that need cleaned up.  Right now, I am trying to find the problematic items by finding those that have led to invoicing errors, but a report that is directed explicitly at finding those items incorrectly edited, would be invaluable!  There are perhaps many items that are problematic that have not yet been invoiced, or we caught before finalizing.  The new report should catch these whereas the old report would not. 

Thanks for the offer of yet another "Really Useful Report"!

Sam

Re: Incorrect prices appearing randomly in invoices

The final (I hope) version of the Invoice Price Check Report is now in the resource library at http://www.openvpms.org/customisation/invoice-price-check-report

 

Regards, Tim G

Re: Incorrect prices appearing randomly in invoices

I'm carrying on with this thread as we are continuing to have issues, despite my having taken steps that I thought would mitigate the issue. 

Tim G's invoice price check report has been invaluable in helping to identify those products that are having pricing issues.  Using that, I have done the following for each problematic product (including the Blood Collection Fee service from the original post!) :

1) Using the copy button, I copied the problematic product to a new product.

2) Closed out the old product with the current date and inactivated the old product.

3) Adjusted the new product starting date to the current date + 1 and removed the '(copy)' text from the new name.

After having been away for about 10 days, I ran the invoice pricing check report again (attached) and see that yet another invoice with the bogus blood collection fee went out in my absence.  The invoice creation function appears to have accessed the same $30 tax-ex price that was sometimes appearing prior to the mitigation. I have attached the details of that invoice as well as screenshots of the old inactivated price and the current active one. 

I am comfortable in believing now that the original issue stemmed from improper re-purposing of items and that the bogus $30 price is not randomly derived but comes from residue of a parent product.  But apparently (and surprisingly!) my 3-step program as described above, has not broken whatever internal links exist to the bogus pricing.

Any suggestions heartily accepted!

Thanks,

Sam 

 

AttachmentSize
Invoice Price Check.pdf 34.21 KB
Invoice.jpg 88.88 KB
BloodCollectionFee-Inactive.jpg 77.55 KB
BloodCollectionFee-Active.jpg 78.18 KB

Re: Incorrect prices appearing randomly in invoices

1. Find all products that have an ex-tax price of $30.00 or $31.50

select e.entity_id, e.name, e.arch_short_name, e.active, pp.product_price_id, pp.arch_short_name, pp.description, pp.start_time, pp.end_time, pp.is_fixed
from product_prices pp
join products p on p.product_id = pp.product_id
join entities e on e.entity_id = p.product_id
where pp.price = '30.00' or pp.price = '31.50';

If you use multiple tax rates, change the above to include each possible tax-ex price.

2. Create a new invoice for the customer and patient, and charge one of the above products.  Switch the product to Blood Collection Fee, and verify the total changes to $25.20. If so, try 3.

3. Create a new invoice for the customer and patient, and charge one of the above products.  Save the invoice. Switch the product to Blood Collection Fee, and verify the total changes to $25.20. If so, try 4.

4. Create a new invoice for the customer and patient, and charge one of the above products.  Click OK, and re-edit the invoice. Switch the product to Blood Collection Fee, and verify the total changes to $25.20. If so, try 5.

5. Repeat for each of the products found in 1.

If the above all behave correctly, then its more likely that the price has been manually changed on the invoice.

 

Re: Incorrect prices appearing randomly in invoices

Tim,

I've done as suggested above on all $30.00 (no tax), $31.50 (5%) and $33.60 (12%) items.  All permutations produced results as one would normally expect.  I don't think this is surprising as the error is certainly not regular, although the error amount is consistent. It could have all 'just worked' this time around.

Attached is an Invoice Price Check report for the Blood Collection Fee from Jan 1, 2016 to today.  The price would have been adjusted on Jan 16, 2016... hence the decrease in UP errors after that time.  You will see however, that since that time, we have had randomly-occurring errors (but always of 31.50 tax-inc).

My belief is that in Jan 2016, Blood Collection Fee was re-purposed from some other item whose price was $30.00.  I don't know of any way to check that however.

There is no possibility that the Blood Collection Fee was manually altered in the invoices.  Particularly the more recent ones, as this has been a significant topic of conversation with all staff.  We've even had staff meetings regarding the need to monitor prices.  But as stated before... in the rush of the day, some items will slip through... as happened just last week (Mar 22 in the report).

I'm thinking that the next step might be to inactivate all problematic items and create entirely new items by hand (no copy) with different names.  I can't think of any other way to make sure that all ties to 'parent' items are truly severed.

Thanks,

Sam

AttachmentSize
Invoice Price Check(2).pdf 71.3 KB

Re: Incorrect prices appearing randomly in invoices

What do you get for:

select e.entity_id, e.name, e.arch_short_name, pp.name, pp.product_price_id, pp.name, pp.description, pp.start_time, pp.end_time, pp.active, pp.price, linked.entity_id, linked.name
from products p
join entities e
    on e.entity_id = p.product_id
join product_prices pp
    on e.entity_id = pp.product_price_id
left join entity_relationships r
    on r.source_id = e.entity_id and r.arch_short_name = 'entityRelationship.productLink'
left join entities linked
    on linked.entity_id = r.target_id
where e.name like "Blood Collection Fee%";        

 

Re: Incorrect prices appearing randomly in invoices

Ok, that's bizarre.

No idea where that $95.24/day is coming from.  Compare the query output to inactivated product 149 pricing details and linked tab...

AttachmentSize
BloodCollectionQuery.pdf 27.85 KB
BloodCollectionFee-ID-149.jpg 103.42 KB
BloodCollectionFee-LinkedTab.jpg 60.28 KB

Re: Incorrect prices appearing randomly in invoices

Sorry - the query is wrong. Mark 2:

select e.entity_id, e.name, e.arch_short_name, pp.name, pp.product_price_id, pp.name, pp.description, pp.start_time, pp.end_time, pp.active, pp.price, linked.entity_id, linked.name
from products p
join entities e
    on e.entity_id = p.product_id
join product_prices pp
    on e.entity_id = pp.product_id
left join entity_relationships r
    on r.source_id = e.entity_id and r.arch_short_name = 'entityRelationship.productLink'
left join entities linked
    on linked.entity_id = r.target_id;

Re: Incorrect prices appearing randomly in invoices

Hmm...

What should I be looking for here?  I get a listing of over 1900 items.  Looks like all of them actually...

All of the linked.entity_id and linked.name in the final two columns are NULL.  I can attach it if you want but it is 106 pages.

Sam

 

Re: Incorrect prices appearing randomly in invoices

Sam - to the best of my knowledge, you setup is 'funny' because of the tax situation. You run two tax rates, PST=7%, GST=5%; you set the tax rates at the product level (not at the product type, or practice level), and some products (Blood Collection is one) have no PST, only GST.

Blood Collection = $24 ex tax, so with GST only we have the inc-tax price of $25.20

Since the tax can be excluded at the customer level, I wondered if the errors were customer linked. No joy - I sorted the Invoice Pricing Check output by customer number and there were multiple sales to the one customer (eg id 4055) with the different prices.

Next try: the tax rate for the $31.50 sale is 31.25% - humm, does not feel as though this relevant. ie if we were picking up the tax rate from a GST+PST product, the tax rate would be 12% and the $24 ex tax becomes $26.88 inc tax.

I don't think that this is getting us anywhere, but if you want me to add a tax column to the invoice pricing check report, I could do so.

Regards, Tim G

Re: Incorrect prices appearing randomly in invoices

Tim...

Yes, funny tax setup, although not unusual for NA and not outside the capabilities of OVPMS I don't think.... even back to 1.7. 

Interesting idea about the errors being related to the customer, but as you say, that does not seem to be the link.  You'll notice on page 7 of my Invoice Price Check(2).pdf, txn 99746 (customer 5995) was properly invoiced, txn 103699 to the same customer was in error, and then when corrected in txn 103728, it invoiced correctly again.  Also, from what I see on the Customer info page, one can tag a customer with tax exemptions, but not tax surcharges as would be required in this case.  

So I totally agree with your analysis... not customer or tax-related.  The Blood Collection Fee issue seems to be connected to picking up some $30 ghost value, perhaps derived from another item (?).

And I'll just throw in (or shoot down) some other possible red-herrings:

1) I do see that contrary to my observation in the very first post, many, but NOT all errors, occur as the first item entered on the invoice.

2) After the Blood Collection Fee price change in early 2016, no errors occurred though the entire year, but we have had three in the past three months.  The first error occurred on Jan 19, 2017, shortly after migrating the database and moving to 1.9.  I will go through my full Invoice Price Check report tomorrow and see if I can detect whether or not the early 2017 date is relevant.   

It's interesting to note from the report that the original Blood Collection Fee (Product #149) showed the error, as does the currently active copy (Product #6598).  And yet one would think that the copy function would not copy any of an item's history.  If it is not in #6598's 'history', then where is the same $30 coming from?

Thanks for the offer of a modified report, but I don't think it would be helpful in this case (not being tax-related) and I can work fine with the existing final version that I have... it's awesome!

Sam

Re: Incorrect prices appearing randomly in invoices

Just thought I should wrap up this thread.  Issue resolved in 1.9.2.  See OVPMS-1891.

On the plus side, we have a much clearer idea as to how to do price changes correctly, and have access to much better logging/reporting. Many, many thanks to the Two Tims!

Sam

Syndicate content