'Optimize query with loads of IN entries
I have quite a massive query that I want to optimize, it consists of 1 table request and 5 table left joins. This query takes 0.3428 sec to complete ( Results: 4,340 total, Query took 0.3428 sec) I am working with about 10000 entries which will definitely grow.
Now the query by it self is not the problem it is the IN statements that is the biggest problem.
- I have 2 IN statements
- Both are in the WHERE statement
- For this specific page load both have a big amount of ID's, 3344 amount of id entries Example: (99, 1, 5, 8458, ...)
- Both IN statements will have the same set of 3344 ID's Example: ((cf.catid IN ( 99, 1, 5, 8458, ... ) AND cf.cid=c.id) OR p.category IN ( 99, 1, 5, 8458, ... ))
The query looks like this:
SELECT
p.id, c.id AS pCid, c.name AS cName, p.name, p.seo,
p.description AS pDescription, cd.description,
p.category, p.archive, cf.catid, cf.pid, p.order_nr,
c.order_nr AS cOrder, c.seo AS cSeo, cat.name AS catName,
cat.order_id, pr.price, pr.sale_price, pr.sale_expiry,
IF( pr.sale_price > 0, pr.sale_price, pr.price ) AS `oPrices`,
pr.member_price, p.`set`, p.get_the_look,
c.from_text_price, c.thumb, c.code AS colour_code,
p.code AS product_code, p.supplier_part_number,
p.oem_part_number, p.make, p.model, p.year, p.sub_model
FROM
products p
LEFT JOIN category_featured cf ON p.id=cf.pid
LEFT JOIN colours c ON c.pid=p.id
LEFT JOIN colour_descriptions cd ON c.id=cd.colour_id
LEFT JOIN category cat ON cat.id=p.category
LEFT JOIN pricing pr ON pr.cid=c.id
WHERE
(
(cf.catid IN ( .. 3344 ID entries .. ) AND cf.cid=c.id) OR p.category IN ( .. 3344 ID entries .. )
)
AND p.archive='0'
AND p.status='1' AND c.status='1'
AND c.archive='0'
AND cat.status IN (1,2)
GROUP BY `c`.`id`
ORDER BY `oPrices` DESC
Is there a better way to do a check for specific ID's in a table using the IN statement or maybe use a different check all together?
Speed is the main issue here, I want to achieve the best performance possible.
So far what I did and how some of the settings are set:
- I created indexes for those tables (only the columns that are INT (integers) that are used in this query have indexes)
- Some tables are MyISAM some are InnoDB (other tables that are not used in the query have a relation with a few tables that are in the query so they had to be InnoDB)
- no relations between the tables in the query exist
- to run the query I use PHP and MySQLI
Thanks
UPDATE!!!!
I noticed why the query is so slow the new column that I create, using the IF statement oPrices
and then useage of "ORDER BY oPrices
DESC" makes the query slow, once I remove it the query only takes 0.00009 of a sec which is amazing!!! But now I wont get the correctly ordered data and if even I do the ordering with PHP I will have to create a new pagination function which is not ideal.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|