Tagged: Filteringlarge data sets
- This topic has 6 replies, 2 voices, and was last updated 10 years ago by
sitesurety.
| Author | Posts |
|---|---|
| October 30, 2014 at 4:38 pm #3547 | |
 sitesurety
Post count: 13
|
I’m looking to build a system that retrieves a set (row) of values where an item matches a filter in an ajax request. i.e. All active bookings, bookings between dates, etc. The input data is regularly updated with a CSV file. I want to make the data visible/editable via WordPress admin area. I wanted to get some feedback if using MasterPress is suitable for this type of use case where: What are people’s thoughts on the best way to achieve this? I could write a custom SQL query and not use master press, but I wanted a data powered solution with a prebuilt API and admin integration already built. |
| October 30, 2014 at 7:06 pm #3549 | |
 traversal
Post count: 207
|
Hey there. If you’re trying to map a single row in the CSV to a single (booking) post, this should be able to be setup with MasterPress successfully. The only thing I’m unsure of in the scenario you’ve described is the round trip after editing fields in MasterPress. Would those then be exported back to a master CSV or something like that? Note that once your field definitions are setup MasterPress uses standard post_meta tables, so you can update the post_meta directly via SQL, and MasterPress should be able to pickup those changes when you edit. MasterPress also stores data like dates etc in ways that should be queryable, so you can use standard WordPress meta queries if the MasterPress API isn’t giving you the results you need. One thing I WOULD avoid is using multiple-item (repeater) field sets for very large data sets attached to just one post. These are harder to update with the API, and they’ll also lead to poor performance in the backend. But if each row corresponds to a single post with a bunch of custom fields then that should be okay. |
| October 30, 2014 at 7:14 pm #3550 | |
|
sitesurety
Post count: 13
|
Thanks for the help. It is more for display purposes only (view only) on the front end. The front end essentially can be thought of as a view only to the primary datasource which comes from the CSV file, and no output again needs to happen. To clarify your comments – I would need to create a post of type ‘booking’ with custom fields for each entry for the fields of data I want.. Then query for the posts matching the filters. (as opposed to one post using a repeater field. I was hoping there would be a more condensed way to do this than create a post for each entry, but from a users perspective it probably makes sense and is easier to manage the entries if they’re laid out seperately. I’m assuming there won’t be any major issues querying for a bunch of posts that match a range and extracting an array or similar of those results to display. Essentially it just needs to be a HTML dump through ajax, updating the view as the query changes with the new results. RE importing data – I’ll probably have to setup a word press cron to run a custom import process every x minutes, which parses the CSV file and creates the posts if they don’t exist (unless I use a third party module for this – Happy if you can recommend one) Thanks again. |
| October 31, 2014 at 8:47 am #3551 | |
|
traversal
Post count: 207
|
Yep I think a post per data row is definitely the most scalable way to do it. There shouldn’t be any issue with querying a bunch of posts on a range, no, apart from working out the correct meta query to do it. Dates are stored as UTC-style format, so they should be able to be used with comparison operators. I don’t know of any plugin specifically to pull in CSV data regularly. But that shouldn’t be too hard to custom code anyway. |
| December 5, 2014 at 1:10 pm #3572 | |
|
sitesurety
Post count: 13
|
Hey there, I’ve found when pulling the date through: i.e. $f->raw() in the MEOW filter function, the result is empty. I was expecting a string with the date time stamp. How do I get the raw value of the datetime field? |
| December 5, 2014 at 1:50 pm #3573 | |
|
sitesurety
Post count: 13
|
Seems I had the wrong pass through value. I’m not sure it does time comparison directly though, as it comes through as a time field.. so I’m updating the function to check for time. |
| December 5, 2014 at 2:57 pm #3574 | |
|
sitesurety
Post count: 13
|
$fieldSetDate = DateTime::createfromformat(“d M Y h:i A”, $fv); // Masterpress field date conversion if ($filterDate instanceof DateTime && $fieldSetDate instanceof DateTime) { |
You must be logged in to reply to this topic.